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ABSTRACT 



This study evaluated the effectiveness of the Crossroads 



Cafe program, an instructional program for adults that promotes a 
nontraditional learning environment with limited intervention by a 
professional teacher. It studied 475 limited-English-prof icient (LEP) adult 
learners using Crossroads Cafe videotapes and text materials under field 
conditions for at least 13 weeks at 10 different sites across New York State. 
The instructional materials included 26 half hour videos supported by 
textbooks and other ancillary materials. The learners were pre-tested and 
then randomly assigned to one of three learning conditions: distance 
learners, LEP adults who worked at home with videos and text materials and 
had occasional telephone and mail contact with a teacher; hybrid learners, 
who worked at home like distance learners but also met weekly or biweekly 
with a teacher; and controls, LEP adults who did not participate in any 
formal English language learning program during the experimental period. Of 
the LEP adults, 57.5% returned for post-testing. The research questions 
included the following: Did the hybrid and distance groups perform 
significantly better than the control group? Did either the hybrid or 
distance group score significantly higher than the other on these tests? 
Ancillary questions inquired if in general any group felt much more confident 
or capable in English. General conclusions were that both the hybrid and 
distance learners achieved significantly more robust gains than the control 
group, though not significantly different scores from one another. Listening 
gains were less impressive. This report provides substantial quantitative 
data backing up its conclusions, including numerous charts, figures, and 
diagrams. (Contains 22 references.) (KFT) 
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CROSSROADS CAFE IMPACT STUDY 



ABSTRACT 



Purpose 

The Impact Study sought to establish the effectiveness of the Crossroads Cafe program by 
studying limited English proficient (LEP) adult learners using Crossroads Cafe videos and text 
materials under field conditions for a period of at least 13 weeks at 10 different sites across New 
York State. Crossroads Cafe, an instructional program for adult learners who lack English language 
proficiency, promotes a non-traditional learning environment with limited intervention by a profes- 
sional teacher. The instructional materials include twenty-six highly professional half-hour video 
episodes supported by textbooks and other ancillary materials. 

Conditions 

For the study, 475 limited English proficient (LEP) adults participated in an orientation 
session, were tested either on the Literacy Skills section of the Basic English Skills Test (BEST) or 
the reading and listening comprehension sections of theComprehensive Adult Student Assessment 
System (CASAS), and then randomly assigned to one of three learning conditions: 

1) Distance Learners — LEP adults who essentially worked at home with the videos and 
text materials, except for regular telephone and mail contact with a teacher, 

2) Hybrid Learners — LEP adults who reviewed their work with a teacher present weekly 
or bi-weekly, but viewed the videos and completed the text material at home, and 

3) Controls — LEP adults who did not participate in any formal English language study pro- 
gram during the experimental period. 

All adult learners were to be pre and post-tested. They completed demographic and person- 
al history questionnaires at the orientation and final sessions of the program. All received stipends 
of $25 at their pre-testing session and were promised $35 at post-testing 26 weeks later. At the 
conclusion of the study, 273 (57.5%) of the LEP adults returned for post-testing. 

Research Questions 

The Impact Study was designed to answer two primary research questions by comparing 
pre and post-test scores and survey data collected from experimental and control groups: 

• Did the hybrid and distance groups perform significantly better than the control group 
on the BEST, CASAS Reading and CASAS Listening tests? 

• Did either the hybrid or distance groups score significantly higher than the other on 
these tests? 

Other ancillary questions were posed such as: Do Crossroads Cafe students feel that they 
understand more English language radio and television? Are they more comfortable speaking 
English on the telephone? Are they more confident using their English language skills? Have their 
job situations changed? 
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Findings — Test Scores 



Analysis of pre and post-test achievement data from the reading components of the BEST 
and the CASAS Battery, as well as Student Performance Level ( SPL) analyses, documented signi- 
ficant and robust gains in performance by Crossroads Cafe learners when compared with gains 
attained by controls during the same period. No differences in reading achievement were found 
between the hybrid and distance conditions. On average. Crossroads Cafe learners gained almost 
twice as many points as controls on the BEST (1 1 versus 6) and a little more than twice as many 
points as controls on the CASAS Reading (1 0 versus 4.5). On the Student Performance Level scale 
(SPL), Crossroads Cafe learners gained approximately twice as much as controls, 1.19 to 0.56 
SPL’s. 

Listening test gains were less robust. Nonetheless, overall hybrid and distance learners 
scored higher on the post listening test than did controls; but only for the distance condition was 
there a statistically significant difference. However, those hybrid and distance LEP adults who re- 
mained in the program for the full cycle of study attained post-test scores significantly higher than 
did controls. 

Findings — Corroborative Data 

Teacher ratings of students’ understanding and verbal expression, students’ assessments 
of improvements in their own communications and every day life skills, and the students’ reports 
of changes in their employment also reinforce the conclusion thatCrossroads Cafe is an effective 
educational intervention for LEP adults under the experimental conditions. 

During the experimental period, approximately 25% of the members of all groups acquired 
new jobs or received raises or promotions. They consistently felt that their new jobs were better 
than their previous jobs. However, hybrid and distance learners more frequently attributed these 
positive events to improved English language skills than did controls. At post-testing, hybrid and 
distance learners also gave significantly higher ratings and more pronounced improvement assess- 
ments to their own comfort levels in using English on the phone and listening to English language 
radio than did controls. 

Implications for Family Literacy Initiatives 

Analysis of the data revealed extensive use of Crossroads Cafe in learning situations involv- 
ing family members, especially children. An unexpected advantage offered by Crossroads Cafe 
may be its use in family literacy initiatives. Based upon these data, further investigation is war- 
ranted for considering the use of Crossroads Cafe as an important intervention in the effort to 
develop full family literacy. 




Page 2 of 2 



8 



List of Graphic Presentations 



Number Title Page 

Figure 1: ESOL Teaching Experience 75 

Figure 2: Learning Activities Prior to Crossroads — Retained Students 76 

Figure 3: National Origin of Pre-Tested Students 77 

Figure 4: First Language of Pre-Tested Students 78 

Figure 5: Years of Formal Education before Emigrating 79 

Figures: Pre-Tested Students by Age 80 

Figure 7: Pre-Test Distribution by Condition 81 

Figure 8; Retention: Urban, Suburban, Rural 82 

Figure 9: Distribution of Retained Students by Years of Education 83 

Figure 10: Retention by Years of Formal Education 84 

Figure 1 1 : Distribution of Retained Students by National Origin 85 

Figure 12: Distribution by Language of Retained Students 86 

Figure 13: Post-Test Distribution by Condition 87 

Figure 14: Student Retention Rates by Age ! . . 88 

Figure 15: Distribution of Retained Students by Age 89 

Figure 16: Residence in the USA in Months 90 

Figure 17: Months at Current Residence 91 

Figure 18: English Spoken Daily by Students 92 

Figure 19: Percent of Pre-Test Students Not Needing to Speak English 93 

Figure 20: Days per Week Read English Language Newspaper 94 

Figure 21: Retained Learners’ Employment Pattern 95 

Figure 22: Years at Current Job 96 

Figure 23: Reasons Cited for Studying English 97 

Figure 24: Post-Test Adjusted Means — BEST 98 

Figure 25: Post-Test Adjusted Means — CASAS Reading 99 

Figure 26: BEST Reading Mean Point Gains 100 

Figure 27: CASAS Reading Mean Point Gains 101 

-i- 



O 



9 



Figure 28: Post-Test Adjusted Means — CASAS and BEST SPL’s 102 

Figure 29; Student Performance Level (SPL) Gains 103 

Figure 30: CASAS Listening Unadjusted Mean Point Gains 104 

Figure 31: CASAS Reading Test Unadjusted Mean Gains by Duration 105 

Figure 32: CASAS Listening Test Unadjusted Mean Gains by Duration 106 

Figure 33: Unadjusted Mean Gains in CASAS Reading 

by Initial Proficiency Level 107 

Figure 34: Mean Ratings by Teachers of Students’ Understanding of Videos 108 

Figure 35: Mean Ratings by Teachers of Students’ Verbal Expression 109 

Figure 36: Coefficient of Correlation between 

Teachers’ Ratings and Pre and Post Test Scores 110 

Figure 37: Unadjusted Mean Gains of Student Ratings 

of Telephone Comfort Level Ill 

Figure 38: Unadjusted Mean Gains of Student Ratings 

of Radio/TV Listening Comfort 112 

Figure 39: Post-Test Student Ratings of Confidence in English Usage 113 

Figure 40; Students’ Perceived Mean Gains in English Proficiency 114 

Figure 41 : Mean Student-Teacher Contact Minutes by Log 115 

Figure 42; Problems Interfering with Studying — All Students 116 

Figure 43: Percent of Students Reported Watching with Families by Log ... 117 

Figure 44 Total Responses All 5 Logs 

Percent of Students Watching Videos with Families .... 118 

Figure 45; Partner Selection by Hybrid Students 119 

Figure 46: Partner Selection by Distance Learners 120 

Figure 47: Choice of Partner — All Students 121 

Figure 48: Employment Changes Attributed by 

Students to Improved English 122 

-ii- 



O 



10 



List of Tables 



title page 

Table 1: Participating Sites and Their Characteristics 13 

Table 2; Retention Rates by Test, Pre-Test Means, and Condition 33 

Table 3: Unadjusted Mean Gains Reading and Listening All Conditions 47 

Table 4: CASAS Reading Test Unadjusted Mean Gains by Duration 48 

Table 5: CASAS Listening Test Unadjusted Mean Gains by Duration 49 

Table 6; Coefficient of Correlation between Teachers’ Ratings 

and Pre and Post Test Scores 54 

Table 7; Multiple Correlations of Perceived Comfort Levels 57 

Table 8: Correlations between Students’ Self Assessment 

and Teacher Ratings — Post-Test 59 



- 111 - 



O 



11 



Crossroads Cafe Impact Evaluation Study 




12 



Crossroads Cafe Impact Evaluation Study 

October 31, 1999 

Purpose 

This report presents the findings, the research protocols employed, a review of the 
data collected, and a description of the statistical analyses performed as part of the Cross- 
roads Cafe Impact Study for which data collection began in March 1998 and concluded in 
July 1 999. Funded by a grant from the New York State Education Department, this Impact 
Study was undertaken by the Center for Advanced Study in Education (CASE) of the City 
University of New York Graduate School to determine the effectiveness of the Crossroads 
Cafe instructional materials in strengthening the English language skills of adults with 
limited English proficiency when used in non-traditional learning modes. 

As part of the NewYork State Education Department’s (NYSED) commitment to the 
development of a strong distance learning adult education program, NewYork State, along 
with Florida, Illinois, and California, entered into a public-private partnership with INTELE- 
COM of Pasadena, California, and with the Heinie and Heinie Publishing Company to 
develop the Crossroads Cafe program. Designed specifically for use by adults with limited 
access to professional instructors. Crossroads Cafe employs sophisticated video and text 
materials to teach English in both hybrid and distance learning modes. 

This Impact Study was undertaken following an earlier formative study’ for which 
data was collected from twenty-four (24) experienced professional ESOL instructors and 
seven hundred fifty-five (755) adult limited English proficient (LEP) students at twenty-two 
(22) sites through a six week summer period in rural, suburban, and urban areas of New 



1 Spiegel and Rayman, 1997 



- 1 - 



o 



13 



York State. The analysis of the collected data revealed a strong positive subjective 
assessment of the effectiveness and the ease of implementation of the Crossroads Cafe 
materials by students, instructors, and program administrators. Partly as the result of the 
favorable findings of the limited formative study, this Impact Study was undertaken to 
assess the effectiveness of Crossroads Cafe over twenty-six weeks, a period more likely 
to reveal a measurable impact of the materials on the English language proficiency of the 
Impact Study’s subjects. 

Need for Expansion of ESOL Services^ 

The United States is a nation of immigrants, admitting as many legal immigrants and 
refugees annually as are admitted by all other countries combined. In 1997, the foreign 
born population of the United States, 25.8 million people or 9.7 percent of the total pop- 
ulation, was larger than at any other time in our country's history. During the decade of the 
1980's, approximately twelve million legal and undocumented immigrants — a record 
number— entered the country, accounting for one-third of the population growth during that 
period.^ 

In the United States, among the immigrant population of approximately 25.8 million, 
approximately half have serious difficulties with English. The 1990 Census reported that 
25.5 million adults, 18 years or over, in this country speak a language other than English 
at home and, of these, an estimated 12 million cannot speak, understand, or write English 



2 Source: NYS Education Department, Office of Workforce Preparation and Continuing Education, Albany, New York, 
1996, except as cited othen/vise. 

3 The Newest New Yorkers 1990-1994, NYC Department of City Planning, 1996, p. 4. 
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without difficulty. Consequently, they are unable to participate fully in American society 
because they lack proficiency in English. 

For the decade of the 1990's, current projections estimate a one-third increase in 
American immigration, bringing the total immigrant population by the year 2000 to more 
than 35 million in a nation of 276 million (12.7%). Two-thirds of these migrants (66. 7%) 
currently reside in only six states.'* Of these, approximately 17.4 million qualify as Limited 
English Proficient (LEP) adults warranting ESOL instruction. Demographers expect that 
by the year 2000, half of the nation’s ten largest metropolitan areas will have majority 
populations whose native language is not English.® 

In an effort to accommodate this burgeoning LEP population, ESOL education has 
now become the fastest growing sector of adult education in the United States. Spurred 
on by the Adult Education Act (AEA) amendment of 1990, the number of ESOL programs 
serving the LEP population increased almost 200 percent in the decade from 1 985 to 1 995. 
Nonetheless, demand for ESOL instruction continues to exceed current service delivery 
capacity, and this shortfall is expected to continue well into the foreseeable future. 

Today, because of limited federal, state, and local budget allocations, classrooms 
are overcrowded, funding is inadequate, and waiting lists are long. Although a substantial 
number of people — approximately 2.3 million — attended ESOL classes in 1997 in the 
United States, this figure represents only 13.2 percent of the 17.4 million people who cur- 
rently require English language instruction. 



4 U.S. Bureau of the Census, “Popciock," September 1999. 

5 U.S. Bureau of the Census, Current Population Reports, March 1997 
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New York State is second in the nation, exceeded only by California, with regard to 
the percentage of its population that is foreign born (19.6%) and its actual number of LEP 
adults who would benefit from ESOL instruction.® Although enrollments in New York State 
ESOL programs have increased dramatically — by more than 73 percent in the past 
decade — ESOL services during that period reached only 633,000 out of approximately 
4.25 million New Yorkers in need.^ 

What is Crossroads Cafe? 

Crossroads Cafe is an instructional program for adult learners who seek to improve 
their English language proficiency. The program promotes non-traditional learning oppor- 
tunities for gaining English language proficiency with limited intervention by a professional 
teacher. The program offers materials that are designed to capture the interest of new 
Americans in an entertaining and informative format through the presentation of twenty-six 
highly professional half-hour video episodes. These videos are supported by textbooks 
and other ancillary materials for use by limited English proficient (LEP) adults and by non- 
professional helpers of their own choosing. 

Crossroads Cafe incorporates features designed to meet the individual needs of a 
broad array of adult learners: 

• A flexible multi-level approach. Crossroads Cafe print materials are designed 
to meet the needs of learners at low-beginning through high-intermediate levels 
of English. Learners work progressively by using the same video series and 
Worktext at each successive level. 



6 U.S. Bureau of the Census, Language Use and English Ability, Persons 18 Years and Older, by State: 1990 Census 

7 NYS Education Department, Office of Aduit and Continuing Education 
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• Context-based learning. Crossroads Cafe story-lines portray situations that are 
relevant and meaningful to a foreign-born population based on the premise that 
the importance of language to learners is determined by the situations in which 
language is used. 

• Self-instructional methods. The collateral print materials were designed for in- 
dependent learners who might not have access to more traditional ESOL 
classes. To ensure that print materials are user friendly, technical language is 
avoided. Self-checks and an associated answer key are provided for each unit. 
Students are encouraged to work with partners. 

• Communication skills. The materials enable users to develop communication 
skills by focusing first on the purpose of communication and then on the forms 
of communication, both written and spoken. 

• Facilitation of different life styles. Activities are designed to appeal to different 
kinds of learners. For example, those who prefer exercises that elicit more ob- 
jective "right-or-wrong" answers may complete the fill-in-the-blank or matching 
questions on sentence sequencing or on reading comprehension. Others may 
prefer the activities that elicit self-expression, the development of fluency, the 
development of meta-cog nitive skills, or reflection on the learning process. 

The Crossroads Cafe ESOL Adult Learning Program consists of five major components; 

Video — The main component is a series of 26 half-hour video episodes that 
provide education through entertainment. The videos, which are closed-caption- 
ed, portray the story of six ethnically diverse characters whose lives intersect at 
Crossroads Cafe, a neighborhood restaurant. Each video episode contains two 
imbedded instructional segments that are directly related to the episode’s theme: 

■ Wordplay, which demonstrates a specific language function (e.g., expressing 
a person’s ability to do something) and the language structures necessary 
to express that function (e.g., can, know how to). 

■ Culture Clips, a documentary style segment, which examines issue-oriented 
social or cultural themes that are dramatized in the story. 

• Worktext — Twenty-six units contain exercises designed to develop story com- 
prehension, language skills, and higher order thinking for intermediate level 
learners. Three graduated levels of exercises are provided, including high be- 
ginning (focusing on words and phrases), low intermediate (focusing on phrases 
and sentences), and high intermediate (focusing on discourse and paragraphs). 

• Photo Stories — Simple dialogue balloons and four color photographs present 
a condensed pictorial tour of each episode for lower level learners. 

- 5 - 
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• Teacher Resource Package — To maximize the benefits of the program, two 
resource books are available for teachers which include general instructions on 
instructional methods, episode-specific suggestions for optional enrichment 
activities, and fifty-two reproducible masters that provide interactive work oppor- 
tunities for students in a hybrid or distance learning program. 

• Partner Guide —A step-by-step manual that is designed for a non-professional 
aide or English proficient friend or family member to help the adult learner learn 
English. 

Research History 

Although piloted as recently as 1995® and initially distributed in 1996, Crossroads 
Cafe has already been subjected to several preliminary studies. Johnston and colleagues 
and Brzezinski and Leitner (1996) quantitatively and qualitatively tested the effectiveness 
of using several teaching models for Crossroads Cafe over a short course of study in highly 
controlled conditions. Findings indicated that all students learned using Crossroads Cafe 
regardless of delivery model, and all liked the course and course materials. McLean 
(1997) reviewed management and training concerns regarding the implementation of 
Crossroads Cafe and offered recommendations. Spiegel and Rayman (1 997) qualitatively 
studied implementation models and surveyed administrator, teacher, and student re- 
sponses to Crossroads Cafe. In that study, administrators described Crossroads Cafe pro- 
gram models as relatively easy to establish, and teachers reported that students spent 
more time "on task." However, these studies reported small sample sizes, short durations, 
limited sampling of the materials, and an absence of comparison groups; therefore, any 
assessment of program effectiveness based upon these findings is inconclusive.® 



8 Johnston. Brzezinski and Stites. 1996 

9 Rudes and Stempteski. 1997 
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Relationship between the State Study and the Federal Study 



The New York State Crossroads Cafe Impact Study was designed to parallel closely 
the "Pilot Test of the Impact Evaluation of the Crossroads Cafe Program" conducted by Dr. 
Blair Rudes, principal investigator at Development Associates, Inc., under contract with the 
U.S. Department of Education.^® To ensure uniformity of practice in both state and federal 
research projects, to broaden the experimental data base, and to facilitate later compar- 
ative analysis of findings, close collaboration was maintained with Dr. Rudes through the 
duration of the Impact Study. 

Sharing information with the Crossroads Cafe federal program, researchers facili- 
tated both studies. As a member of the New York State Impact Study Advisory Committee 
since its inception. Dr. Rudes provided expertise in research design, in data collection form 
construction, and in determining questionnaire format. These materials, developed by 
CASE for the state Impact Study, were also incorporated into the federal study protocols. 
Dr. Rudes’ team also benefitted from CASE’S New York State experience derived from the 
formative study” and from the CASE Impact Study’s early findings. 

Research Questions 

This Impact Study was designed to establish the effectiveness of Crossroads Cafe 
by studying a relatively large sample of adult learners under field conditions for a period 
up to twenty-six weeks, by employing most of the Crossroads Cafe support materials, and 
by comparing pre and post-test results of hybrid and distance learning groups with data 
collected from control groups. 

Rudes, Zehler, Hopstock and Stephenson, 1999. 

1 1 Spiegel and Rayman, 1997 
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The Impact Study was designed to answer two primary research questions: 

• Did the hybrid and distance groups perform better than the control group on 
the BEST and CASAS Reading and CASAS Listening Tests? 

• Did either the hybrid or the distance groups score significantly higher than 
the other on these tests? 

A number of other ancillary questions were posed: 

o Is time on task a predictor of student achievement? Did time on task 
change as the course continued? 

o What is the relationship between amount of teacher contact and student 
performance? 

o Do Crossroads Cafe students feel that they understand more English lan- 
guage radio and television following their Crossroads experience? Are 
they now more comfortable speaking English on the telephone? Are they 
now more confident using their English language skills? 

o Did their job situations change either during or after their Crossroads 
experience? 

How Was the Study Conducted? 

Impact Study Advisory Committee ^^ 

An Impact Study Advisory Committee comprised of faculty members of the City 
University of New York Graduate School and other experts in research design, statistical 
analysis, and ESOL instruction developed an Impact Study research design to assess the 
effectiveness of Crossroads Cafe materials in both hybrid and distance learning modes. 
To execute the research plan formulated by the Advisory Committee, the CASE Cross- 
roads Cafe professional staff structured specific research protocols and designed a series 



12 Members of the Advisory Committee: Dr. Alan Gross, Dr. Bert Ftugman, Dr. Ruth Diones, Dr. Susan Stempteski, Dr. 
Patricia Mooney Gonzalez, Dr. Carole Tittle, Dr. Dolores Perrin, Dr. Blair Rudes. 
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of data collection instruments for use before, during, and after the Impact Study experi- 
mental period. 

Experimental and Control Conditions Defined 

All subjects participating in the study were randomly assigned to one of three condi- 
tions: 1) Distance Learners, 2) Hybrid Learners, or 3) Control. For the Impact Study, these 
conditions were defined by their learning activities: 

1. Distance Learning Condition 

a. Learners attended an orientation session to learn about Crossroads Cafe 
and provided personal and demographic information; 

b. Adult learners were identified as lacking English language proficiency; 

c. Learners underwent pre and post-testing; 

d. Learners worked with Crossroads Cafe primarily on their own; 

e. Learners maintained contact with their teachers only via telephone and/or 
mail throughout the learning period. 

2. Hybrid Learning Condition 

a. Learners attended an orientation session to learn about Crossroads Cafe 
and provided personal and demographic information; 

b. Adult learners were identified as lacking English language proficiency; 

c. Learners underwent pre and post-testing; 

d. Learners worked with Crossroads Cafe primarily on their own; 

e. Learners met weekly or bi-weekly with their teachers in a limited aca- 
demic relationship. 

For the Impact Study, distance learning and hybrid conditions were considered ex- 
perimental modes. Pre and post-test data collected from students and teachers working 
in the two experimental modes of instruction were compared with data collected from con- 
trol subjects. The control condition was defined as follows: 

3. Control Condition 

a. Adult learners underwent the same pre and post-testing as those in 
Hybrid and Distance Learning conditions; 
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b. Adult learners were identified as lacking English language proficiency; 

c. Adult learners provided the same personal and demographic information 
at the beginning and at the end of the 26 week period as those in Hybrid 
and Distance Learning conditions; 

d. Adult learners did not participate in any formal English language study 
program during the 26 week experimental period. 

In all three conditions, a student self-reporting questionnaire was administered, and 
pre and post-testing was conducted. Forthe hybrid and distance learning groups, Student 
and Teacher Logs were completed at predetermined intervals throughout the study. Com- 
pleted logs were forwarded to the CASE Crossroads Cafe office for scanning into the com- 
puter SPSS program.’^ Students participating in the study received stipends of $25 at pre- 
testing and $35 at post-testing. 

Non-Traditional Learning Strategies 

In an effort to reach the multitude of unserved LEP adults, hybrid and distance 
learning modalities are being employed experimentally nationwide as alternatives to tra- 
ditional classroom instruction.’'’ Initially conceived as models that can more easily accom- 
modate convoluted adult student schedules, hybrid and distance learning alternatives may 
also reduce the need for new classroom construction, a benefit not to be ignored during 
times of limited budgets. Although teacher student contact time is reduced as compared 
with traditional classroom models, time on task, individual pacing, and other factors which 
are part of a well-structured Crossroads Cafe program may compensate for less teacher 
student contact and, thus, sustain a satisfactory rate of English language acquisition. 



1 3 Statistical Program for the Social Studies, SPSS 8.0 for Windows 



14 Rudes and Stempleski, 1997 



Additionally, the inherent flexibility of hybrid and distance learning enables adult 
learners to pursue their studies systematically at home at their own convenience and at 
their own pace. Not only may this flexibility reduce the student attrition customarily found 
in traditional ESOL programs, but these innovative strategies may provide instruction for 
LEP adults who othenwise would not have an opportunity to learn English in a structured 
environment. 

Rudes and Stempleski have identified characteristics of successful distance learners 
in programs other than ESOL.^® In doing so, they noted the remarkable similarity of those 
characteristics with the characteristics of the ESOL adult population. In both populations, 
the students are older and highly motivated; most are women with small children; most are 
employed either part or full time; most are burdened with multiple obligations and suffer 
limited resources. Researchers were hopeful that those characteristics that characterize 
successful distance learners in non-ESOL programs would also be operative among adult 
ESOL hybrid and distance learners. 

Use of the BEST and CASAS Battery 

Based on a search of the literature and because a federal research group had 
already indicated a preference for the Basic English Skills Test (BEST), the Impact Study 
design also included the BEST as its pre and post-testing instrument.^® Pre-testing began 
in March 1998. However, within the first month, very high pre-test scores on the BEST 
were reported, raising concerns of a potential "ceiling effect" for BEST post-test scores. 
Moreover, as earlier studies had indicated that Crossroads Cafe materials were most effec- 



15 Rudes and Stempleski, 1997 

16 The Basic English Skills Test (BEST), Test Manual, Center for Applied Linguistics, Washington, DC., 1984 
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tive with intermediate and advanced students, pre-selecting subjects with low pre-test 
scores would generate a subject pool least appropriate for the research purposes of this 
study. Therefore, rather than mitigating a ceiling effect by including only students whose 
scores on a scale of 0-77 were 50 or lower, the Comprehensive Adult Student Assess- 
ment System (CASAS) battery was adopted. 

CASAS offers a wider testing range than the BEST and is available in scaled forms 
for multiple levels of difficulty. Adults accepted with BEST scores remained in the study. 
All students who subsequently became part of the study were tested on the CASAS 
battery. Thus, the effects of Crossroads Cafe are reported in terms of two different 
achievement tests. 

Dr. Rudes, principal investigator for the federal pilot study, was apprised of CASE’S 
findings regarding the pre-test results; and, discovering the same problem with the BEST 
in the federal pilot, he also substituted the CASAS Reading Life Skills section as the feder- 
al pilot assessment instrument. 

With the substitution of the CASAS, researchers realized that a comparison of BEST 
and CASAS test results on samples of adult learners in different Crossroads Cafe condi- 
tions was possible. Therefore, the research team requested permission from the NYS 
Education Department to increase the numbers in the Impact Study so that the number 
tested on the CASAS battery would approximate those tested on the BEST. In summary, 
an opportunity was presented to study the impact of Crossroads Cafe as assessed by two 
different standardized tests commonly used in the ESOL community and to compare their 



17 Comprehensive Adult Student Assessment System (CASAS), San Diego, CA., 1993. 
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findings. The team considered that the findings generated by the pre and post-test forms 
of each instrument could help to determine those conditions under which each test might 
be used most appropriately. 

Which Were the Participating Sites? 

Ten publicly funded organizations and agencies participated in this Impact Study. 
Six Community Based Organizations (CBO’s) in four of the five boroughs of New York City 
(Brooklyn, Bronx, Manhattan and Queens) provided the urban experimental component. 
The remaining four sites were Rochester, Buffalo, Long Beach, and Mt. Vernon, New York. 
Table 1. Participating Sites and Their Characteristics^^ 



Site 


Experimental 

Conditions 


Location 


Number of 
Teachers 


Number of 
Cycles 


Bronx Educational 
Services 


Control 

Hybrid 

Distance 


City 


2 


1 Full 
1 Short 


Chinatown 

Manpower 


Control 

Hybrid 


City 


2 


1 Full 


YMCA Elesair 


Hybrid 


City 


1 


1 Short 


Flatbush 

Haitian 


Control 

Distance 


City 


1 


1 Short 


Frontier 


Control 

Distance 


Rural 


1 


1 Full 
1 Short 


Highbridge 


Control 

Hybrid 


City 


1 


1 Full 


Long Beach 


Control 

Hybrid 


Suburban 


2 


2 Full 


Mt. Vernon 


Control 

Distance 


Suburban 


1 


1 Full 


Rush-Henrietta 


Distance 


Rural 


1 


2 Full 


Shorefront 

YM-YWHA 


Control 

Hybrid 


City 


4 


1 Full 
1 Short 


TOTALS 


8 Control 
6 Hybrid 
5 Distance 


6 City 

4 Out of City 


16 Teachers 


10 Full 
5 Short 



Full cycles are defined as 26 weeks or more; short cycles are defined as fewer than 26 weeks but at least 13 weeks. 
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Each site offered instruction in only one experimental condition, except for Bronx 
Educational Services which provided instruction in both hybrid and distance conditions. 
Thus, six sites conducted hybrid programs; five sites conducted distance learning pro- 
grams; and eight of the ten sites provided control students. 

Given historically^® projected levels of student attrition, the loss of one participating 
site, and the small number of hybrid students who initially pre-tested on the CASAS test, 
the need for an expanded student base was soon apparent. Upon request, three site 
directors agreed to initiate a second series of experimental student groups, and one 
additional site, YMCA Elesair, was enlisted to replace the Father Billini Center in Queens. 
Duration of the Experiment 

Of the fifteen Crossroads Cafe programs that were conducted at the ten sites, ten 
completed full twenty-six week cycles. However, in an effort to expand the experimental 
base within realistic time constraints, five programs that started late were concluded before 
twenty-six weeks elapsed. In Table 1, these programs are designated short cycle pro- 
grams, although all were operational 16 weeks or more. 

Who Were the Instructors? 

Eighteen teachers were trained in the use of Crossroads Cafe materials, in distance 
and hybrid instructional techniques, and in the implementation of the research protocols.^® 
Procedures for pre and post-testing were reviewed and sample video and text materials 



19 Young, Morgan, Fleischman, and Fitzgerald, 1994. Forty*three percent of students are lost after 3 months of study, and 
57% of students are lost after 6 months of study. 

20 Training curriculum attached as Appendix A. 
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were distributed. The teachers and program administrators were briefed in the research 
protocols^’ and instructed in the proper use of the data collection instruments. 

During the course of the Impact Study, two instructors left because of changes in 
employment. Of the remaining sixteen instructors, twelve were women. All were experi- 
enced ESOL teachers. Thirteen could converse in a language other than English. 

Teachers underwent an average of 20 hours (std = 14.59) of training in Crossroads 
Cafe, which included preparation for this Impact Study as well as for prior work with Cross- 
roads. Among the teachers in the study, 1 1 taught in the hybrid condition, 4 taught dis- 
tance learners, and one taught both. (See Figure 1: ESOL Teaching Experience.) 

Of the instructors participating in this Impact Study, two who were teaching in the 
hybrid condition had prior experience in both distance and hybrid situations. Six teachers 
in the hybrid condition had prior experience teaching in that mode. Three distance learning 
instructors had prior experience as distance learning teachers. Five had ESOL teaching 
experience in traditional classroom settings only. During the course of the Impact Study, 
Crossroads Cafe teachers worked with an average of 22 students at a time, with the num- 
bers ranging from 5 students to 70. All teachers received an honorarium for participating 
in the study. 

Teacher Training 

All Impact Study teachers participated in a six hour Crossroads Cafe professional 
staff training session that was led by three experienced Crossroads Cafe teachers, one of 
the co-authors of the Crossroads materials, and the research staff. The importance of the 



21 Protocols employed attached as Appendix B. 
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training session was stressed because of the indications regarding the value of the teacher 
training that surfaced in the Crossroads Cafe study conducted in Florida.^^ The training 
included the following activities; 

1 . A detailed discussion of the Crossroads Cafe materials, 

2. A presentation of the philosophy behind the Crossroads Cafe instructional 
program, 

3. An analysis of the specific language skills targeted by each of the Cross- 
roads Cafe components, 

4. An exposition of the art of successful Crossroads Cafe hybrid and distance 
teaching, 

5. A discussion of successful teachers’ stratagems that help to reduce adult 
ESOL student attrition, 

6. A thorough explanation of the research protocols and how they were to be 
applied, 

7. A dissection of all of the data collection forms to familiarize teachers with 
their content, function, and importance, and 

8. An explication of the components of the BEST, how to proctor the test, and 
how to score it.^^ 

Limited Engiish Proficient (LEP) Adults 

In the United States, the distinguishing characteristics of the limited English profi- 
cient population are revealing:^'* 

a Virtually all are foreign born; 

a Just over half are women; 

a Two-thirds are between the ages of 22 and 45; 



22 McLean, Butler, Jameson, J. Shaffer and R. H. Shaffer, 1997 

23 See Appendix A. 

24 Rudes and Stempleski, 1997 



- 16 - 



o 



28 



D Forty-six percent (46%) are employed; 

D Sixty-five percent (65%) earned at least a high school diploma in their native 
land; 

D Eighty-five percent (85%) currently live in major metropolitan areas; 

D Sixty-nine percent (69%) of those in ESOL classes speak Spanish; 

D Twenty-seven percent (27%) speak an Asian language; 

D Fifty-six percent (56%) reside in states west of the Mississippi River. 

Characteristics common to ESOL learners tend to increase the poignancy of their 
plight.^® Surveys conducted by researchers^® in the field report that these adult learners 
are strongly motivated to learn English. They seek to participate more fully in American 
life. They want to play a more important role in their children’s education. They feel great 
pressure to improve their work situation or to get a better job. They know that mastering 
the new language will enable them to manage everyday chores better. They recognize a 
need to help others in their native land. They are aware of the degree to which enhanced 
English proficiency will boost their own self-esteem. Yet, only to a fortunate few have 
access to professional English language instruction. 

The Benefits of English Language Instruction 

As a matter of public policy, limited English proficiency among a large segment of 
the nation’s population denies all Americans the enhanced quality of life that would accrue 
from the talents, knowledge, culture, and skills that are contributed by new Americans. The 
inability to read, write, and speak English also imposes a cultural and ethnic isolation upon 
new Americans that impedes their contributions to and their absorption into American 
society. The dearth nationwide of ESOL instruction perpetuates economic dependence 
among immigrants to the detriment of us all. 

25 Rudes and Stempleski, 1997 

26 Spiegel and Rayman, 1996; Brod, 1995 
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Our nation, traditionally, has recognized the value of the contributions made by im- 
migrants in all fields of endeavor. As compared with access to citizenship in most other 
nations, access to full citizenship in the United States throughout the nation’s history has 
been relatively easy. Virtually open borders through the eighteenth, nineteenth, and most 
of the twentieth centuries are indications of the value our nation has placed upon absorp- 
tion of immigrants. To this end, the nation’s constitution and laws apply equally to all 
residents regardless of status. They guarantee every individual and sub-group without 
qualification equal opportunity, economic stability, and fundamental political protections. 

To facilitate the entry of new Americans into mainstream American life, traditional 
ESOL instruction generally incorporates into the curriculum essential pre-employment, 
workplace, vocational, and citizenship survival and life skills. Yet, although adult learners 
in ESOL classes recognize the value of their ESOL education, real life complications, over 
which they have little or no control, often thwart adult learners’ efforts to attend class 
regularly, if at all. 

Student Attrition 



Causes of student attrition among ESOL adult learners frequently include personal 
problems, such as demands of a job, scheduling, transportation, child care, and illness. 
In addition, invasive activities at initial classroom encounters, such as batteries of tests and 
extensive questionnaires, may discourage continued attendance; yet, agency reporting and 
documentation requirements as well as research methods often require initial assessment 
and data collection to validate and fund programs. 
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Certainly, outreach to new Americans and to others who lack English language pro- 
ficiency is consonant with professed public policy priorities'^ and, therefore, warrants spe- 
cial efforts to ensure success. Crossroads Cafe programs may be the means by which 
professionals in adult education can address the problem of student attrition, possibly the 
most difficult of issues to resolve. Because of its design and flexibility as an instructional 
instrument. Crossroads Cafe provides an attractive vehicle by which ESOL program admin- 
istrators and teachers may be able to increase enrollment, retain adult learners, provide 
the support services students need, facilitate independent study, and create a dynamic 
interactive communications regimen with students studying independently. Creative use 
of Crossroads Cafe offers the promise of reducing student attrition to tolerable levels.^® 
This Impact Study has sought to assess the effect of the Crossroads Cafe hybrid and dis- 
tance learning programs and methodologies on student attrition. 

Design and Procedure 

This study implemented a Pre-test — Post-test Control Group Design utilizing three 
groups: control, hybrid and distance learning. All adult learners were tested with either the 
Literacy Skills section of the Basic English Skills Test (BEST) or the reading and listening 
comprehension sections of the Comprehensive Adult Student Assessment System 
(CASAS). Each student completed a demographic and personal history questionnaire. 
Hybrid and distance students also completed personal performance logs on five units at 
regularly spaced study intervals. Their instructors completed logs rating their students’ per- 
formance at the same intervals. 

27 Adult Education Act (AEA) amendment of 1 990 

28 Brod, 1997; Fitzgerald, 1997 
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Experimental Protocols 



The research design was implemented through the application of detailed protocols 
which instructed the professional staff regarding the conduct of the Impact Study. The pur- 
pose of the protocols was to ensure uniformity of conditions across sites, to provide precise 
directions for Impact Study implementation by the professional staff, to create clear expec- 
tations for each condition, and to limit experimental error as much as possible. 

Hybrid Condition : Students working in the hybrid condition studied, reviewed, and 
used Crossroads Cafe videos and support materials at home as described previously. 
Teachers met with students either weekly or bi-weekly to monitor the students’ progress, 
to provide encouragement for their continued studies, to converse with learners in English, 
to discuss Worktext or Photo Stories exercises, to have students write, read aloud, work 
on pronunciation, and explore English idioms. The sessions were primarily oral inter- 
changes. When feasible, students met in groups and worked cooperatively on program 
units. If students had special needs or if they fell behind, teachers had the flexibility to 
work with them individually. Teachers phoned or wrote to students who had not contacted 
a teacher for two weeks. 

Distance Learning Condition : Students working in the distance learning condition 

studied, reviewed, and used Crossroads Cafe videos and support materials at home. 
Teachers contacted students by telephone only at least bi-weekly. Students were urged 
to call instructors as often as they wished, but to call at least once a week. If students 
were not in contact for two consecutive weeks, teachers mailed letters of inquiry encourag- 
ing students to continue with their studies. To facilitate communications between student 
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and teacher, voice mail capability was established for all distance learning teachers. Each 
adult ESOL student was encouraged to find a partner with stronger English language skills 
to help him/her work on the Crossroads Cafe units. The partner was urged to be present 
during the student’s phone sessions with the instructor. A letter with basic unit-related 
questions, study suggestions, and words of encouragement accompanied all video tapes 
mailed to students. 

Another goal in establishing the protocols was to ensure full usage of all five compo- 
nents of the Crossroads Cafe program. Teachers and students were exhorted to avoid 
contact with non-Crossroads Cafe ESOL materials during their Crossroads Cafe studies. 
Students were taught how to study using Crossroads Cafe. Students were also en- 
couraged to watch the videos frequently and to work with a partner whose English profi- 
ciency surpassed their own. They were urged to work at comfortable levels in the Worktext 
initially and then to work at graduated proficiency levels as they progressed through the 
Worktext. Teachers suggested that students schedule regular weekly study time and to 
view videos with their families. Students were expected to complete at least one unit every 
two weeks; however, individual differences resulted in a wide range of unit productivity. 
Student Orientation 

Teachers and researchers participated in the effort to recruit students and helped 
to arrange the initial orientation meetings. Appendix A presents the recommended activi- 
ties for the introductory session at all sites at which pre-tests were administered and initial 
background information forms completed. Following the pre-testing and distribution of the 
Crossroads Cafe materials, students in all three conditions received their stipends. If time 
allowed, controls were dismissed and Video #1 was shown to the experimental group. 
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Measurement Tools 



The underlying purpose of collecting multiple measures of the construct “student 
success” is triangulation. That is, each of the measures described below assesses a 
unique component of “student success.” The results converge to create the desired con- 
struct. In this study, “student success” is defined in the following terms: 

1 ) Improved language skills and comprehension as measured by higher scores 
on standardized tests, 

2) Higher teacher ratings of student understanding, 

3) Higher student ratings of their own degree of understanding of English 
language radio and television, 

4) Increased confidence and comfort in speaking English on the telephone as 
measured by students’ higher ratings of their own facility, 

5) Positive growth in employability as measured by advances in job status, and 

6) Advances in job status attributed to improved English language skills. 

Instrumentation 

Several data collection forms^® were designed: 

1. Initial Student Background Information Form, 

2. Student Log, 

3. Teacher Log, 

4. Follow-up Student Information Form, and 

5. Teacher Background Information Form. 

Initial Student Background Information Form 

The Initial Student Background Information Form, in addition to compiling student 
demographic data, also recorded the following student ratings: 

29 

See Appendix C for a complete set of forms. 
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o How confident did they feel when using English? 
o How comfortable did they feel when speaking English on the telephone? 
o How well did they understand English language radio and television? 
o How many days per week did they read an English language newspaper? 
o Could they get by in their communities without speaking English at all? 
o If they did speak English during the course of a regular day, approximately 
how many hours were spent speaking English? 
o Were they currently employed? 
o Did they have young children at home? 
o Did they live in a multi-generational household? 

In addition to the original questions, the follow-up form completed at the end of the 
study also asked students to indicate any changes that had occurred in their work status 
during the experimental period. 

Student Loos 

Upon the completion of five consecutive units, students submitted a Student Log for 
the fifth unit completed. Therefore, during the program, five logs evenly interspersed 
throughout the 26 week period were completed. For each unit, students recorded time on 
task in hours and minutes watching the video, working on the Worktext and/or Photo 
Stories, and working with a partner. Students also identified relationships with their 
partners (e.g. a friend or relative) with whom they studied and reported whether or notthey 
had watched the video with their families. Students also entered into the logs the total 
number of times they viewed each particular episode. 

Teacher Logs 

Teacher Logs were completed to parallel each of the 5 logs completed by the stu- 
dents. Teacher Logs also recorded the number of contact hours the teachers spent with 
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each student. On the Logs, teachers assessed each student’s general understanding of 
the video and the student’s ability to communicate verbally that understanding. A five point 
rating scale was used, in which "5" indicated excellent understanding of the video or excel- 
lent verbal expression of understanding, and "1" reflected hardly any understanding of the 
video or very poor verbal expression of understanding. Finally, teachers noted in their 
Logs any problems the students reported that interfered with their studying. 

FoIIow-ud Student Background Information Form 

Atthe post-testing session, students also completed Follow-up Student Background 
Information Forms which provided post-course ratings on the level of confidence students 
felt in using English, their comfort level in speaking English on the telephone, and their per- 
ceived level of understanding when listening to English language radio and television. The 
form also asked the students to rate how much their English language skills had improved 
during the experimental period and to report all changes that had occurred in their employ- 
ment status during this period. 

Teacher’s Background Information Form 

The Teacher’s Background Information Form provided data regarding the teachers’ 
professional experience in ESOL teaching and their experience with Crossroads Cafe in 
particular. 

Tests Employed in the Impact Study 

Basic English Skills Test (BEST) 

Originally, student pre and post-testing to determine English language proficiency 

achievement was to be assessed using the Basic English Skills Test (BEST). When the 
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BEST was discontinued as the assessment instrument and the Comprehensive Adult Stu- 
dent Assessment System (CASAS) was selected as the alternate, the Research Advisory 
Committee recommended that, although the BEST had its limitations, valuable information 
could be gathered by post-testing on the BEST those who had pre-tested on the BEST. 
As both tests are useful in ESOL instruction and are widely accepted instruments in the 
ESOL arsenal, comparative data generated by the two instruments were likely to be of 
academic and practical value. 

The BEST Literacy Skills section including reading and writing and the CASAS read- 
ing and listening comprehension sections are standardized and criterion referenced. The 
BEST was developed in the 1980's to assess "survival level competencies" of newly 
arrived refugees. 

Topic areas ... were personal identification, greetings, kinship terms, health terms, 
parts of the body, numbers, time, money, shopping for food and clothing, housing, 
emergencies, directions, using the telephone, completing simple forms, writing 
checks, addressing envelopes, and other similar writing activities. Grammatical 
structures identified as necessary for the accomplishment of these tasks included the 
simple present and present progressive tenses "yes/no" and "wh-" questions, and 
negations. Language function given top priority included imparting information, 
seeking information, and seeking clarification. (BEST Test Manual p. 53) 

Designed for beginning to low intermediate learners of English, the BEST scores 

can be equated with Student Performance Levels (SPL’s) as defined by the Mainstream 

Language Training Project in 1985 and are, therefore, criterion referenced.^® The SPL’s 

describe specifically, in operational terms, the student’s reading and writing skill levels and 

correlate those skills with typical instructional levels. For example, a student at SPL-V (5) 



30 While BEST is designed for beginning to low intermediate learners. Crossroad Caf6 is meant for advanced 
beginners and intermediate students. This is not the best match. 
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can "read and understand some short simplified ma\.er\a\s related to feas/c needs with some 
misinterpretations [their italics]."®^ 

Despite varying item types such as Fill in the Blank, Multiple Choice, and Open 
Ended Writing Tasks, the internal consistency reliability estimates of Form B and Form C 
of the BEST, the forms used for pre and post-testing in this study, are both 0.97. The 
BEST was group administered, and students wrote their answers directly in the test book- 
lets. Two Crossroads staff members were trained to score the two essay questions. The 
inter-rater agreement was high (r = .99). 

Comprehensive Adult Student Assessment System (CASAS) 

Like the BEST, the CASAS Life Skill series focuses on assessing competency by 
the use of test items that focus on context based "functional life skills." Unlike BEST, 
CASAS offers an entire program, which includes an appraisal of initial skills, a selection of 
competencies to be learned, targeted instruction, and an evaluation of outcomes appropri- 
ate to the original needs assessment. In this way, instruction and assessment are linked 
within the curriculum. 

The explicitly defined competencies needed by adults to function in everyday life, 
as defined in Uodhcutt’s Adult Performance Level Study {CASASTechn\ca\ Manual, 1993) 
are integral to the CASAS battery. CASAS competencies are hierarchically structured: 

1 . Content priority areas are listed; 

2. For each content area, competencies are defined; 

3. A measurable competency statement for each competency is provided; and 

4. Each competency may be presented in several item formats or tasks, e.g. 
filling in blanks or reading a chart. 



31 SPL’s are defined in Appendix D as cited in the BEST Manual, pages 66-73. 
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CASAS competencies can also be correlated with the Secretary of Education’s 
Commission on Achieving Necessary Skills (SCANS) competencies. These competencies 
qualify CASAS as a criterion referenced test. An advantage in using the CASAS battery 
is that it measures four overlapping levels of progressively increasing difficulty. Descrip- 
tions of typical real-life skills are associated with each skill level and with each scoring 
range from Level A (beginning literacy) through Level D (pre-GED).^^ 

The CASAS battery provides conversion tables that translate raw scores on all four 
levels into scores scaled within the limits of a single range. Therefore, with a common 
reference base, comparisons of scores can be made easily, reliably, and without regard 
to the forms used in testing. With CASAS, an ESOL instructor or researcher can assess 
a wide range of students’ reading comprehension levels in less than one hour and their 
listening comprehension levels in about forty minutes. The overlapping forms of the 
CASAS minimize the likelihood of a "ceiling effect." All items are multiple choice and 
answers are recorded on a separate answer sheet. Both the listening comprehension and 
reading sections can be group administered. KR reliability of the anchoring forms ranges 
in the vicinity of .85. 

Historically, using commercially available tests such as BEST and CASAS to assess 
adult ESOL learners’ English language proficiency has been suspect because researchers 
have recognized the fact that students lacking English proficiency may not be able to re- 
spond satisfactorily to multiple choice paper and pencil tests: 

o They may have difficulties with reading or with the vocabulary; 



32 See Appendix E for skill level descriptions as defined for the CASAS battery, 
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o They may be unfamiliar with cultural assumptions underlying an item format; 

o They maybe inexperienced in test-taking, test differences, or testing conven- 
tions, e.g. not knowing how to fill in an answer sheet.^^ 

Nonetheless, with these caveats in mind, performance by experimental condition 
among three randomized cohorts in which the weaknesses inherent in paper and pencil 
tests are operative equally throughout. In addition, ancillary data provided by teacher and 
student logs provide alternative collateral corroborative information on student achieve- 
ment, student confidence and self-esteem, and changes in the student’s life situation. 

In reviewing the BEST and the CASAS tests, Sticht (1990) expressed concerns 
about the CASAS battery. He concluded that the CASAS reading test, while representing 
useful life-skills, assesses a comparatively low level of reading ability. Sticht found CASAS 
questions to require literal comprehension only and exclude higher-order comprehension 
and thinking skills, such as drawing inferences or making evaluations. The BEST was sub- 
ject to the same findings. 

For the group administered Listening Comprehension Test, researchers must con- 
sider issues specific to that genre. The CASAS Listening Comprehension Test, like others 
of its kind, relies upon the subject’s ability to read printed components of the test items 
while listening to spoken information. The subjects are required to read maps, to read 
signs, to track visual test items in test booklets as they listen to directions, and then to 
enter responses to directions or questions on a printed answer sheet separate from the 
test booklet. These multiple tasks and mixed competencies certainly confound the 
researcher’s ability to assess a subject’s listening comprehension level. The effort to 

33 Burt and Keenan, 1994. 
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assess a singular listening ability requires the successful application of multiple commun- 
ication modalities. Therefore, any simple and direct assessment of a subject’s listening 
comprehension based upon this group administered listening comprehension test is 
severely compromised, if not entirely precluded. Corroborative data from other sources 
must be sought before firm conclusions can be drawn when assigning a listening skill level. 

Lytle and Wolfe (1989) noted the small number of items used to sample CASAS 
competencies and pointed out that the focus solely on competencies placed limits on the 
instructional and learning fields. Davis and Yap (1992) reported that students felt that 
CASAS tests underestimated the actual level of the students’ own achievements. 

Despite these concerns and limitations, the Impact Study Advisory Committee 
deemed the CASAS and BEST to be the most appropriate tests of those available for the 
evaluation of Crossroads Cafe. With their focus on real-life situations, these tests are well 
suited to the context based Crossroads Cafe format. 

Also the Worktext/Photo Stories print materials emphasize multiple levels of difficulty 
and address mastery of grammatical structures within the communicative context of lan- 
guage function. This stress on mature life experience learning within a graduated adult 
language acquisition program reflects the dominant focus of both the BEST and the 
CASAS battery. 

Results 

Data Collection 

Prior to the implementation of the experimental model and application of the proto- 
cols, researchers had anticipated a straightfonward assessment of the Crossroads Cafe 
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hybrid and distance learning programs. However, at the very first introductory session, ad- 
justments were required to adapt the protocols to the realities of the field conditions. 

Although plans had called for group pre-testing of ESOL adult learners at one or 
perhaps two sessions at each site, many adults who were eager to participate found that 
because of extenuating circumstances they were unable to attend the scheduled sessions. 
Children were sick; promised transportation did not materialize; unexpected job-related 
responsibilities arose. Many ESOL students requested a delay in starting. On average, 
for the originally scheduled pre-test date, as many as sixteen of twenty-five ESOL learners 
expected did not appear. 

Many who did appear, of necessity, brought small children with them. At sites in 
Queens, Long Beach, Rush Henrietta, the Bronx, Brooklyn and Westchester, several 
children — from infants to about ten years of age — were present during the introduction 
and pre-testing. Other accommodations to care for these children were unavailable, al- 
though in Long Beach and Rush Henrietta special child-care staff had been hired. This ar- 
rangement was only partially successful as funding constraints limited the number of adults 
who could be hired for this purpose and children were often unwilling to remain with 
unfamiliar caretakers. 

Adherence to the Protocol 

As described earlier in this report, teachers were trained in the Impact Study’s 
protocol and were given a detailed Protocol Manual. In these ways, the meaning of 
"hybrid" and of "distance" learning was defined in great detail. While teachers were given 
some flexibility and discretion, they were required to and did adhere to the protocol specifi- 
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cations. For the most part, teacher contact in distance learning was limited to the tele- 
phone. However, students may have visited a site to exchange videos. 

Hybrid meetings were opportunities for students to ask questions, to practice con- 
versation and pronunciation skills, and to write English. Only in one case did the teacher 
report showing some of the videos at the hybrid meetings. While learning was to be self- 
paced and the videos could be watched in any order, some sites proceeded through the 
program in numerical order. Lesson 1 to Lesson 26. 

Controls were asked on the Follow-up Student Background Information Form what 
they had been doing to learn English during the past 26 weeks. As requested of them, 51 
of 53 control students had not taken formal instruction. Most had practiced English by 
listening to the radio, watching television, and speaking English with friends and acquaint- 
ances. ('See Figure 2: Learning Activities Prior to Crossroads as Reported by Retained 
Students.) 

Characteristics of Pre-tested Subjects 

Four hundred seventy-five (475) students were pre-tested for this study. Reflective 
of the population of New York State, the Impact Study participants were extremely diverse. 
(See Figure 3: National Origin Pre-Tested Students.) Students represented 48 different 
nations of origin with fluency in 24 different first languages. The most common first lan- 
guage among the students was Spanish (60.0%); Russian (14.2%), and Chinese (9.7%) 
in that order. (See Figure 4: First Language of Pre-Tested Students.) This distribution re- 
flects the proportions of immigrants arriving in the United States in the 1 997 Census data. 
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Among the participants, 49% of students had attended high school or had 
completed their high school education, (x = 1 1.04 years, std = 3.29). Almost one quarter 
(24%) had attended or completed an institution of higher education. (See Figure 5: Years 
of Formal Education before Emigrating.) Virtually all students could read and/or write in 
their first language — 98.9% and 96.6% respectively of those responding. As is common 
among this population of learners, approximately two-thirds of the students were female 
(66.5%). Most (61%) were between 30 and 49 years of age, averaging of 40.5 (std = 
1 1 .4). (See Figure 6: Pre-Tested Students by Age.) Of the 475 pre-tested participants, 
327 (68.8%) were New York City residents; 148 (31 .2%) were from suburban or rural sites. 

To the degree feasible, students were randomly assigned to experimental condi- 
tions. However, sites varied in the rigor of random assignment. Often spouses were un- 
willing to be separated. Some applicants were adamant about beginning to study immedi- 
ately and refused placement as controls. Pre-tested learners were actively recruited for 
inclusion in the control condition if, after assignment to an experimental condition, they did 
not sustain minimal contact with their teachers. By the start of the program at the fifth site, 
students were being randomly assigned to condition before orientation and testing. Con- 
trols were invited fortesting on a different date to reduce the likelihood that students would 
complain about condition assignments. The distribution by condition of the pre-tested stu- 
dents was as follows: 1 69 were assigned to the hybrid condition; 1 32 were assigned to the 
distance learning condition, and 156 served as controls. (See Figure 7: Pre-Test 
Distribution by Condition.) 

Those enrolled as hybrid or distance learning students immediately received all 
videos and textbooks free. Control students received the no textbooks or materials, but 
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were promised priority placement for study at post-testing, thus establishing an active wait- 
ing list for the next session. All received their stipends of $25 at their pre-testing session. 

Student Retention 

Retention among adult students is a serious concern. In this study, a student is con- 
sidered "retained" if she or he has been both pre and post-tested. This section discusses 
common programmatic occurrences that may relate to attrition and presents data examin- 
ing possible relationships between student background characteristics and retention. 

Of the four hundred seventy-five (475) adults with limited English proficiency who 
were pre-tested, two hundred seventy-nine (279) or 57.3% returned for post-testing — a 
respectable retention percentage for this population.^'* When applied to the entire study, 
the retention rate was better than originally expected. However, variability of retention rate 
across experimental sites and among different ethnic groups presents the likelihood that 
geographic differences and differences in learning styles have a direct effect on attrition. 
Ameliorating these differences may require greater flexibility in retention policies and more 
aggressive outreach practices than can normally be provided in traditional classrooms. 
Table 2. Retention Rates by Test, Pre-Test Means, and Condition 
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Retention rates were lower among those who pre-tested on the BEST. These stu- 
dents were among the first to enter the program and were among the population for which 
the possible shortcomings of the BEST as a pre and post-test instrument were discovered. 
Recognizing the possibility of a strong "ceiling effect," the research staff’s first response 
to the unexpected high scores on the BEST pre-test was to exclude from the study a 
significant number of candidates with higher pre-test BEST scores. 

However, since the Crossroads Cafe program was initially designed for use with 
advanced beginners through high intermediate levels of English proficiency among ESOL 
students, this pre-selection of lower level participants was recognized to be counter pro- 
ductive early in the study. Nonetheless, by selecting candidates in this way, the lowest 
scoring ESOL candidates dominated at the first six sites. Two hundred ten students (21 0) 
were pre-tested on the BEST before the CASAS battery could be deployed. 

To avoid skewing the experimental population, the Impact Study Advisory Committee 
and the research staff responded by adopting the CASAS battery. Because of the multiple 
levels available with the CASAS battery, fears of a "ceiling effect" disappeared. Therefore, 
adults who were pre-tested on the CASAS represented low, intermediate, and, to a lesser 
extent, high levels of English proficiency in experimental and control conditions. 

This change had a direct impact on the percent of students retained. At sites which 
lost most of their experimental learners, the hybrid students had been tested on the BEST. 
Sites employing the CASAS Reading Test show higher retention rates than for students 
tested on the BEST in both experimental conditions. Of those tested on the CASAS 
Reading, 69.6% were retained; of those tested on the BEST, 50.7% were retained. 
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The hybrid students tested on the BEST who did not remain in the program averaged 
a score of 39.5 as compared to an average score of 50.8 for who were retained. This 
difference of 11 .3 points at the start, equal to one full Student Performance Level grade 
(SPL), may reflect the early effort at the first sites to select only those candidates whose 
scores on the BEST were low. Since Crossroads Cafe was designed for LEP adults with 
a higher initial English language proficiency, Crossroads may be an inappropriate interven- 
tion for this population. The higher rate of attrition among these low scoring LEP adults 
may be attributed to a heightened sense of frustration caused by Crossroad Cafe’s diffi- 
culty level. 

The difference in retention rates between the BEST and the CASAS controls runs 
counter to the data regarding the experimental subjects. (See Table 2, page 33.) As 
controls were absolved from any formal language acquisition intervention, the data seem 
to warrant further investigation to seek a better understanding of the behavior of ESOL 
control groups in experimental formats. 

in addition to the initial selection of lower scoring pre-tested BEST subjects, the 
unanticipated cessation of all distance education programs at Southern Westchester 
BOCES left thirty Crossroads Cafe students with no teacher and no program. Not until the 
Mt. Vernon School District agreed to accept the Crossroads Cafe program and not until the 
program was physically housed at the Mt. Vernon Adult Learning Center about two months 
later was the program revived. Other sites suspended operations during the summer. Two 
teachers left the study to take better jobs elsewhere. In one situation, the departure of the 
teacher resulted in the abandonment of the site. Other interruptions common among 
ESOL programs also caused delays and upheavals. 
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Retention and Associated Factors 



Overall, the reasons for student attrition in this study were hiultiple and diffuse. 
Typical programmatic issues contributed to student losses. Students with the lowest initial 
English language skills were more likely to discontinue. Data also suggest differential loss 
depending on cultural influences. However, while several indications for causes of attrition 
were present, too often programmatic, background, and instructional issues were suf- 
ficiently confounded so that drawing firm conclusions or making inferences was difficult. 

In an effort to understand retention as a function of one or more of the following 
student variables, logistic regression analyses®® were conducted seeking correlations: 

1] Number of years studied in school, 

2] Presence of young children in the household — yes/no, 

3] Presence of adult relatives living in the household — yes/no, 

4] Current employment status (a paying job) — yes/no, 

5] Student’s age, 

6] Number of years the student had been living in the United States, 

7] Urban, rural, or suburban site of instruction, 

8] Female or male, 

9] Country of origin, 

10] Pre-test score, and 

11] Experimental condition. 

For BEST students, several of these variables were statistically related to retention: 

■ The relationship between pre-test score and experimental condition were 
statistically significant, with lower scoring hybrid students more likely to drop out 
of the study.®® 

Rural and suburban students were less likely to be retained. The sudden loss of the 
Westchester BOCES, a suburban site, and its delayed reinstatement at Mt. Vernon two 



After initially entering pre-test scores and experimental condition, the additional variables were submitted in a forward 
stepwise selection method based on the probability of a likelihood ratio statistic. 

36 Even with this relationship, the study’s main research questions can be analyzed and unbiased estimates calculated 
using listwise deletion for missing data. That is, all students without post-test scores were dropped from the analysis. 
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months later may have lowered the suburban students’ retention rate. (B = 1 .46)^^ (See 
Figure 8: Retention by Urban, Suburban, and Rural Sites.) 

■ Students with young children were more likely to remain in the study (B = .93). 

■ As years of schooling increased, the student was less likely to be retained. 
However, the relationship between these variables was small (B = -.17) and can 
be discounted. 

■ Based on country of origin, students of Slavic, Eastern European, or Hispanic 
heritage were more likely to leave the study. 

■ Students from the Dominican Republic (46.1%), Haiti (40.0%), and Central 
America (44.9%) showed the poorest retention rates. 

Eastern European student retention rates might have been more robust; but one site 
with mostly a hybrid Slavic/Eastern European population dismissed its students prior to 
post-testing. Efforts to retrieve and post-test these students were unsuccessful so that, 
although twenty-six weeks had elapsed, the post data and retention statistics were lost. 

For students tested on the CASAS Reading and Listening battery, years of school- 
ing and experimental conditions were significantly related to retention. As with those tested 
on the BEST, with increased education the student’s likelihood of leaving increased. How- 
ever, as B weights were small, -.16 and -.14 for CASAS Reading and Listening respec- 
tively, a firm relationship was not established. (See Figure 9: Distribution of Retained Stu- 
dents by Years of Education; and Figure 10: Retention by Years of Formal Education.) 

Among all of the students, those of Chinese origin attained the highest retention rate 
(85%), far outdistancing the nearest best performance by Puerto Rican and Mexican stu- 



37 Unlike a correlation coefficient, B weights can be larger than 1 . 
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dents by more than 18 percentage points. (See Figure 11: Distribution of Retained Stu- 
dents by National Origin; see Figure 12: Distribution by Language of Retained Students.) 
Focus Group Insights 

The Impact Study teachers and administrators convened in June 1 999 for a "Focus 
Group" discussion oftheir experiences with Crossroads Cafe. Retention was an important 
subject. In their view, teacher commitment to the program, persistence, and flexibility were 
key to limiting attrition among LEP adults. Teachers spoke of the need to phone students 
regularly at times convenient for the students. Although the teachers all sought to encour- 
age independent learning among their students, the teachers found that they could not wait 
for students to reach them. 

To ensure continued attendance among their students, teachers found that per- 
sistence in outreach was essential. They initiated contact with their students and often 
went to unusual lengths to ensure timely delivery of Crossroads materials. They main- 
tained a vigorous outreach program both by phone and mail to encourage and cajole stu- 
dents into continuing their studies. In the distance learning condition, the teachers often 
became counselors, friends, and confidants oftheir students. 

Retention, to some degree, is dependent upon the quality of instruction that accom- 
panies Crossroads Cafe. The Focus Group spoke about the importance of teacher train- 
ing. The consensus was that teachers must be "savvy" in their work with Crossroads. 
They must be able to configure the Crossroads materials for their students in their unique 
situations, to recommend specific strategies for successful independent learning, and to 
make deft assessments of their students from afar. 
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Their professionalism must be enriched by a finely honed sensibility to communi- 
cation that is constrained by the telephone or by very limited personal student contact. 
Distance and hybrid teachers need to recognize and respond to messages that are 
inadequately verbalized so that they can provide assistance when the student needs help. 
The Focus Group was unanimous about the need for special training for Crossroads Cafe 
teachers who plan to work in distance and hybrid modes. They all agreed that traditional 
classroom experience, while useful, fell far short of adequate professional preparation for 
teachers in the experimental modes. 

The Focus Group also cited the need for proper orientation for students to Cross- 
roads Cafe. The concept of teaching students to teach themselves is not only foreign to 
most teachers, but the idea often frightens and discourages LEP adults. Crossroads stu- 
dents need to see Crossroads as a friendly, supportive program which will not fail them in 
their effort to master a new language. The LEP adults need to master first the techniques 
for learning on their own, so that learning English is a natural outgrowth of their efforts. 
The orientation session is the first thrust in that direction. The teachers then must follow 
with consistent efforts to improve students’ techniques for self-instruction through the use 
of carefully planned exercises and appropriate materials. Training for the student is, 
obviously, no less important than training for the teachers. 

Retained Student Descriptors ^^ 

Of the 273 retained students, 66% were female; however, despite the large ratio of 
women to men, the retention rate for both groups was essentially the same, 56.3% of the 

38 

These statistics and the percentages that follow are based on responses provided by students to questions on ihelnitial 
Student Background Information and Follow-up Student Information Form. All students did not respond to all questions. 
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females and 58.5% of the males completed the course. The 273 participants who re- 
mained in the study were distributed fairly evenly over the three conditions: control had 89, 
hybrid had 100, and distance had 84. {See Figure 13: Post-Test Distribution by Condition.) 
This distribution may be compared with the pre-test distribution, (See Figure 7: Pre-Test 
Distribution by Condition.) 

The pre-test and post test numbers and percentages of students retained by test 
and by condition are charted in Table 2 on page 32. Retention percentages by test are 
addressed there. 

Similartothe intake group, retained students were diverse, coming from 33 different 
nations (See Figure 11: Distribution of Retained Students by National Origin), speaking 1 8 
different primary languages. (See Figure 12: Distribution of Languages of Retained Stu- 
dents) Among the students, 49.6% had attended high school; the average number of years 
of school was 1 0.86 (std = 3.27). (See Figures 9 and 10.) Some students did indicate that 
they had had virtually no formal education at all while all learners who responded to these 
noted that they could read (100%) and write (98.4%) in their first language. Differences 
among the age groups in percent of retention appear to be minimal with 62.7% of students’ 
ages in the 30-49 year old range. The average age was 39.95 (std = 11.11) and ranged 
from 1 9 to 74 years. (See Figure 14: Student Retention Rates by Age, and see Figure 15: 
Distribution of Retained Students by Age.) 

Of the 273 students who were post-tested, 189 (69.2%) were from New York City 
and 84 (30.8%) resided in suburban or rural areas. Students participating in this study 
reported that they had lived in the United States for an average of 7.5 years (std = 8.23); 
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however, more than half resided in the United States fewer than 2 years, (See Figure 16: 
Residence in the USA in Months.) Although the adult learners reported that they had lived 
at their current address for an average of 4.43 years (std = 4.99), the median time at the 
latest address was also less than two years. The maximum duration reported was 43.5 
years in the United States and 28 years at the same location. (See Figure 17: Months at 
Current Residence.) 

Among the Impact Study post-tested subjects, 147 (53.9%) indicated that they live 
in a community in which they need not speak English at all. Of those who used English 
during the day, the students reported a median of only 2 hours daily in which they spoke 
English in their communities (mean = 3.5, std = 3.58.) (See Figures 18: English Spoken 
Daily by Students; and Figure 19: Percent of Pre-Test Students Not Needing to Speak 
English.) This information was consistent with and tended to corroborate other evidence 
of low daily English usage by urban LEP adults who were primarily working in the hybrid 
condition. Although most students indicated that they read an English language news- 
paper some days of the week, 88 (35.8%) reported that they did not read any English 
language newspaper at all. (See Figure 20: Days per Week Read English Language 
Newspaper.) 

Many of the adult ESOL students reported bearing major family responsibilities. 
Among post-tested participants, 176 (64.6%) were responsible for young children, and 122 
(44.8%) lived in multi-generational households. 

Of the total 273 students post-tested, 142 (51.9%) were working at a paying job. 
Of the 142 who were employed, 86 (60.5%) worked full time, i.e. 31-48 hours per week; 
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43 of the 142 LEP adults (30.3%) worked part time, i.e. 30 or fewer hours each week; and 
12 (8.5%) of the 142 students reported working regularly 49-60 hours per week. (See 
Figure 21: Retained Learners Employment pattern.) The average work week for all 
students was 17.64 hours (std =19.04). 

Most students in fact had not been in their jobs long. Of the 133 responding to this 
question, 91 (68.4%) had been employed at their current jobs two years or less. The most 
common response regarding duration of current employment was one-quarter of a year. 
More than half had been at the same job fewer than two years. However, the responses 
in this category ranged from one month to 28 years. (See Figure 22: Years at Current Job.) 
All students who had reported working more than 60 hours per week on the Initial Student 
Background Information Form left the study prior to post-testing. 

Students who remained forthe entire experimental period reported that, priorto their 
work with Crossroads, they had learned English in various ways. Figure 2 reveals that 
most gained their limited English facility primarily by listening to English language radio and 
television and by speaking English. Hybrid students indicated much more frequently than 
control and distance students that they had studied English in class. The greater fre- 
quency of prior classroom experience among hybrid students may be the result of prog ram 
recruiting methods conducted at hybrid program sites. (See Figure 2: Learning Activities 
Prior to Crossroads as Reported by Retained Students.) 

Among the students who completed the program, the most frequently mentioned 
reasons for studying English were the desire to earn more money and to be better equip- 
ped to accomplish everyday life chores. Students also cited the need to speak English 
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with their children and to provide them more assistance. (See Figure 23: Reasons Cited 
for Studying English.) 

Test Results 

The Impact Study sought to answer two primary research questions: 

1] Did the hybrid and distance groups perform significantly better than the 
control group on the BEST, CASAS Reading, and CASAS Listening tests? 

2] Did either the hybrid or distance groups score significantly higher than the 
other on these tests? 

In seeking answers, an Analysis of Covariance (ANCOVA) procedure which statistic- 
ally adjusts mean post-test scores by taking into account pre-test scores, was applied.^® 
BEST and CASAS Reading Scores 

On both the BEST and the CASAS Reading Test, hybrid and distance students out- 
performed the control students as shown in the post-test adjusted mean gains by partici- 
pants. (See Figure 24: Post-Test Adjusted Means - BEST; and see Figure 25: Post-Test 
Adjusted Means - CASAS Reading.) These differences were statistically significant (t = 
3.9, p<.000 and t = 4.84, p<.000 respectively). 

For the BEST, Crossroads Cafe post-test learners scored an adjusted average of 
59.6 with controls scoring an adjusted average of 52.6 points. Similarly, on the CASAS 
Reading, distance learning students averaged 220.8 points (adjusted), hybrid students 
averaged 219.2 points (adjusted), and controls averaged 213.4 points (adjusted). 



39 Given the specific nature of our questions, a priori contrasts were used. Results were reported using t statistics. 
Controlling for differences in pre>test by using an ANCOVA and using a priori contrasts boosts the statistical power of the analyses; that 
is, it is more likely that real differences can be observed. Tests were also run checking the assumptions underlying inferential statistics. 
For BEST, CASAS Reading and CASAS Listening, the lack of fit tests, used "to check if the relationship between the dependent 
variables and independent variables can be adequately described by the model (SPSS Base Manual)," failed to reject the null hypothesis 
that the model fits (i.e. it fits). Using the Shapiro-Wilk test, the normality assumption was upheld except for CASAS Listening. However, 
"inferences made about means ... are also valid even when the forms of the populations depart considerably from the normal, provided 
the n in each sample is relatively large (Hays, 1981 , p 347)." In addition, only for CASAS Reading was the Levene Test of Equality of 
Variances rejected. 
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student gains are apparent as unadjusted pre and post test means. On average, 
Crossroads Cafe experimental learners gained a little less than twice as many points as 
controls for the BEST (1 1 versus 6) and a little more than twice as many points as controls 
for CASAS Reading (1 0.3 versus 4.5). fSee Figure 26: BEST Mean Reading Point Gains; 
and see Figure 27: CASAS Mean Reading Point Gains.) 

Student Performance Levels Defined 

According to information supplied by the Center for Applied Linguistics (CAL), pub- 
lisher of the BEST and by CASAS personnel, these gains are of statistical and practical 
significance. Center for Applied Linguistics data indicate that 120 to 225 hours of instruc- 
tion can be expected to generate a gain of one Student Performance Level (SPL).'‘° On 
the BEST'*^ a gain of 11 points is the equivalent of one SPL and represents "real student 
achievement." Similarly, CASAS personnel report that a gain of 4 CASAS points has prac- 
tical significance.'*^ For example, in the State of California a gain of 4 CASAS points (1 
SPL) on the reading test is required to demonstrate sufficient improvement to warrant state 
reimbursement. 

BEST. CASAS, and SPL Scores 

Although equating the BEST scores with CASAS Reading Test scores using psy- 
chometric methodologies was not feasible, both BEST and CASAS Reading scores were 



An SPL, Student Performance Level, describes in operational terms the student’s reading and writing skill levels as 
defined by the Mainstream Training Language Project for the BEST. Test items can be scaled on the basis of SPL criteria. 
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E-mail from Carol van Duzer. She does note that gains can depend on a variety of program and student-related factors, 
including intensity of instruction, class size, training and experience of the teachers, and so on. These results are based on field testing 
at 1 9 sites in the early 1 980’s. 
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Personal Communication Linda Taylor, Senior Training Consultant, CASAS. 
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convertible to a common SPL metric. By exercising this conversion, the BEST scale (0-77) 
and CASAS scale (165-231) were reduced to a common seven (7) point range. In per- 
forming this reduction at a ratio of approximately ten to one, some sensitivity in 
measurement was lost. However, by combining the BEST and the CASAS populations, 
the N’s were more than doubled. This much larger base strengthened the power of the 
analysis by increasing the N’s to the extent that gains for all students could be compared 
on the same metric with a high degree of confidence. 

As before, an Analysis of Covariance was applied to these data. As hypothesized, 
distance and hybrid students achieved significantly higher SPL’s than controls (t = 6.16, 
p<.000). For controls, the adjusted mean was 5.2; for hybrid the adjusted mean was 5.8; 
and for distance, the adjusted mean was 6.0. (See Figure 28: Post-Test Adjusted Means 
CASAS and BEST SPL’s.) Again, Crossroads Cafe learners gained approximately twice 
as much as controls, 1.19 to 0.56 SPL’s. (See Figure 29: Student Performance Level 
Gains.) Crossroads Cafe experimental learners in both experimental groups gained on 
average more than one SPL, although distance and hybrid scores did not differ from each 
other statistically. Findings in this analysis using SPL results are consistent with and re- 
inforce the findings of the analyses using the individual BEST and CASAS Reading metrics 
separately. 

CASAS Listening 

CASAS Listening Test scores present gains of a lesser magnitude than those attain- 
ed on the reading test. Applying an Analysis of Covariance, only distance students were 
found to have achieved significantly higher scores than did controls (x = 211.9 adjusted 
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for distance students versus x = 207.3 adjusted for controls (t = 3.43, p<.001). The 
adjusted means for hybrid and control students were virtually the same: x = 206.9 
adjusted for hybrid students, x = 207.3 adjusted for controls. The difference in performance 
between hybrid and distance was pronounced (t = 2.95, p<.004). Only the gain by the 
distance learners of 3.72 SPL points approached the full CASAS 4 point threshold on the 
listening test. (See Figure 30: CASAS Listening Unadjusted Mean Point Gains) 

This outcome was unexpected. Several factors could have affected these findings. 
For example, a larger proportion of hybrid students were from sites within the city in which 
the need to speak English most of the day was minimal, and, therefore, opportunities to 
practice English were limited. Although hybrid students had more structured opportunity 
to practice communicating in English, distance students’ superiority in listening comprehen- 
sion might have been a result of their exclusive reliance upon the telephone. This depend- 
ence may have been instrumental in honing their listening skills to a degree seldom de- 
manded of students in the hybrid mode. 

As noted earlier, scores on the listening test are subject to question because of the 
complexities inherent in a paper and pencil listening instrument. Moreover, a careful 
review of the administration of the CASAS listening test raises questions regarding the 
test’s use. At several sites, the field conditions that were encountered interfered with the 
students’ ability to hear the spoken directions and the accompanying audio tapes. While 
these conditions may have been consistent at both pre and post testing sessions and 
across conditions, the quality of the data collected was compromised. 

The data also indicated that students with low pre-test CASAS Reading and BEST 
scores were more likely to withdraw from the study than those whose pre-test scores were 
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high. This tendency was also operative in the listening component. Distance students who 
remained averaged at pre-testing 210.5 CASAS points. Those who left the study averaged 
only 206 points on the CASAS Listening at pre-testing. This 4.5 CASAS Listening mean 
point difference equals more than one full Student Performance Level (SPL). ('See Figure 
30: CASAS Listening Unadjusted Mean Point Gains; See Table 2: Retention Rates by 
Test, Pre-Test Means, and Condition, page 33) 



Table 3. Unadjusted Mean Gains Reading and Listening All Conditions 



Student 

Gains 


Pre BEST 
Reading 


Post 

BEST 

Reading 


Gain 


Pre 

Reading 

CASAS 


Post 

Reading 

CASAS 


Gain 


Pre 

Listen 

CASAS 


Post 

Listen 

CASAS 


Gain 


Hybrid 


50.75 


61.23 


10;4S, 


205.02 


215.98 


11^6 


203.14 


205.29 




Distance 


49.12 


60.97 




215.00 


224.61 


iili 


210.53 


214.25 


||3^^ 


Control 


44.62 


50.74 




208.89 


213.40 




206.69 


207.10 





Time in the Study 

Originally, all experimental sites were scheduled to run for 26 weeks. However, this 
standard was difficult to sustain because of delayed starts at some sites and because of 
the need for increased numbers necessitated by the switch from the BEST to the CASAS. 
The summer hiatus, honored at many agencies, and the constraints imposed by NY State 
Education Department deadlines also limited the duration of the experimental period. Of 
the 15 participating experimental programs, ten ran for the expected 26 weeks or more; 
however, five programs were concluded prior to reaching their 26 week or 6 month goal. 
These five programs all ran for a minimum of thirteen weeks, some ran longer, but none 
to full term. 
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All short cycle programs involved only students tested on the CASAS Battery, as 
BEST students were all among the first cohorts to enter the study and the full 26 week 
period was available for their programs. Only after BEST was replaced by CASAS and the 
need for additional subjects became apparent was a second initiative undertaken to in- 
crease numbers in the experimental conditions. These second initiative programs are 
those designated here as short cycle. 

In reviewing the data collected from the fifteen sites, the ANCOVA analysis revealed 
that studying English with Crossroads Cafe even in a curtailed program enabled experi- 
mental students to score statistically significant greater gains on the CASAS reading bat- 
tery than did controls (x = 220 adjusted means for short cycle students versus x = 21 3.37 
adjusted for controls; t = 5.14, p<.000). 

Unexpectedly, though, the data also revealed that on the CASAS Reading test, 
gains made by both full and short cycle groups when analyzed statistically did not differ 
significantly from each other (x = 219.9 adjusted for full term students versus x = 220.5 
adjusted for short cycle students; see Table 4 below). Both short and full cycle LEP adults 
showed double the reading gains of controls. Short cycle students gained 9.61 CASAS 
points, and full term students gained 10.76 CASAS points. These mean gains are to be 
compared with control students, whose mean gain was 4.51 CASAS points, or less than 
half. (See Figure 31: CASAS Reading Unadjusted Mean Gains by Duration.) 

Table 4: CASAS Reading Test Unadjusted Mean Gains by Duration 



Duration of Program 


CASAS Pre- 
Test Reading 


CASAS Post-test 
Reading 


Mean Gain 
Reading 


Full Program 


212.93 


222.54 


9.61 


Short Cycle 


207.43 


218.19 


10.76 


Control 


208.89 


213.40 


4.51 
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Table 5: CASAS Listening Test Unadjusted Mean Gains by Duration 



Duration of Program 


CASAS Pre- 
Test Listening 


CASAS Post-test 
Listening 


Mean Gain 
Listening 


Full Program 


209.00 


215.08 


6.08 


Short Cycle 


205.03 


206.07 


1.04 


Control 


206.69 


207.10 


0.41 



A One-Way ANOVA procedure was applied to the CASAS listening data to clarify 
the CASAS Listening ANCOVA results. This procedure compared gains attained by the 
controls with those attained by short cycle hybrid, full cycle hybrid, short cycle distance, 
and full cycle distance students on the CASAS Listening Test. The analysis confirmed that 
although both full distance and full hybrid students gained significantly more on the Listen- 
ing test than did controls (t = 3.18, p<.002 and t = 2.29, p<023, respectively), short cycle 
student gains were not significantly greater than were the control gains. 

CASAS listening adjusted post-test means for short cycle learners and controls did 
not differ statistically (x = 207.8 versus x = 207.3). These data suggest that language 
listening comprehension skills are acquired more slowly than English language reading 
proficiency. These outcomes also help to explain hybrid students’ small gains on the 
CASAS Listening Test. (See Table 5 and Figure 32: CASAS Listening Test Unadjusted 
Mean Gains by Duration.) 

In the data reviewed regarding short cycle learners, the test scores seemed to 
indicate that English comprehension in reading seemed to be acquired more rapidly at the 
start of the program than later. The "learning curve" started at a steep slope and had 
flattened sufficiently by the thirteenth week to approximate the levels attained by the full 
cycle learners. These full and short cycle comparative data raise important questions 
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regarding the most efficient use of Crossroads. The questions raised by these test data, 
supported by teacher assessments of student progress, warrant further study of compar- 
ative implementation of Crossroads Cafe interventions overtime. 

Of the 1 00 students at urban sites, 85 were hybrid; of those, 38 (45%) were enrolled 
in short cycle programs. Conversely, fewer than half, 7 of the 15 distance learners (48%), 
were located in urban sites, and 3 (20%) of the distance learners were in the shorter cycle. 
CASAS Listening test scores at urban sites were significantly lower than at rural or sub- 
urban sites (t = 3.68, p<.000). The adjusted post-test means on the CASAS Listening were 
X = 207.6 for urban students as opposed to x = 214.8 for rural and suburban students. 

The poorer development of listening comprehension skills among hybrid students 
may be attributable to the combined effect of 1) a short cycle which limited the exposure 
to English usage, 2) an urban population within which hybrid students were much less 
likely to need to converse in English to "get through" each day, and 3) the extreme urban 
ethnic density situation reported by many hybrid learners that required no need for English 
at all to participate fully in the subject’s immediate neighborhood each day. When assess- 
ing acquisition of language listening skills among experimental subjects, the impact of the 
conditions of local English daily use, which vary greatly from rural to urban sites, cannot 
be discounted. 

Submitting data that seemed relevant to the acquisition of listening comprehension 
skills to a chi-square analysis revealed a significant relationship between students’ re- 
sponses of "yes or no" to their need to speak English and the characterization of their com- 
munities as urban or suburban/rural (^^ = 5.46, p<.02). This relationship can be inferred 
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also by the frequency with which urban participants reported that they lived in communities 
in which they had no need to speak English during the day. ('See Figure 18: English 
Spoken Daily by Students.) 

Moreover, newest immigrants seem likely to reside in these urban homogeneous 
enclaves and have less need and fewer opportunities to speak English during the day. 
With listening comprehension skills apparently requiring a longer period of time to acquire 
among urban ESOL populations and with the isolation that is too often imposed upon 
urban adult ESOL students, these findings suggest that urban ESOL programs need to 
provide extended periods of time for practicing both listening and speaking English. 
Gains Based upon Pre-Test CASAS Levels 

In an effort to determine optimum use of the Crossroads Cafe materials on the basis 
of students’ starting levels, students were divided into low, middle, and high groups on the 
basis of their initial CASAS Reading pre-test scores. Those scoring below 200 were cate- 
gorized as the "low group;" those scoring between 201 and 220 were categorized as the 
middle group; and high students were those scoring 221 and above. 

For CASAS Reading, those who pre-tested low made the largest gains when com- 
pared to controls. Though not as large, high learners gained more than those in the middle 
group. (See Figure 33; Unadjusted Mean Gains in CASAS Reading Points by Initial Profi- 
ciency Level.) On the CASAS Listening Test all participants categorized as "low" made 
equal gains regardless of condition. For middle level learners, only distance students had 
positive gains on the listening test. Insufficient high level listening test data precluded 
drawing additional inferences. (See Figure 30: CASAS Listening Unadjusted Mean Point 
Gains) 
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The association between initial level of English proficiency and gains made during 
the experimental period warrants further study. As shown in the retention data, those who 
scored very low did not remain in the program. Yet, those whose scores were only some- 
what higher achieved the greatest gains in both experimental conditions. Crossroads 
Cafe’s strength may lie in the power of its initial impact on LEP students, whose threshold 
levels of English meet the minimal standards for use of Crossroads. 

Teacher Ratings 

For each of five units of study, designed to be evenly inter-spaced overthe 26 week 
period, teachers were asked to assign each student two ratings: 

■ How well did the student understand the video, and 

■ How able was the student to express verbally his/her understanding of the video. 

A five point rating scale was used, in which "5" indicated excellent understanding 

of the video or excellent verbal expression of understanding, and "1 " reflected hardly any 
understanding of the video or very poor verbal expression of understanding. During the 
26 week period, teachers’ ratings were expected to provide a record of the rate of improve- 
ment in the students’ understanding of the videos and their ability to express verbally that 
understanding both for hybrid and distance learners. Of course, logs were not completed 
for or by controls. 

Analyses of ratings by teachers of students’ work over five time periods using a 
repeated measures general linear model found that teachers’ ratings of students’ un- 
derstanding of the Crossroads videos did on average increase over time (F = 4.82, 
p<.001 ). Teachers also found that students’ ability to express verbally their understanding 



of the videos improved over time (F = 6.57, p<.000). Further, both of the trend lines 
followed a linear path of growth (F = 11 .02, p<.002 and F = 1 3.84, p<.000) for both under- 
standing and verbal expression. (See Figure 34: Mean Ratings by Teachers of Students’ 
Understanding of Videos; and Figure 35: Mean Ratings by Teachers of Students’ Verbal 
Expression.) 

However, when these ratings were examined by condition, the trend for distance 
students continued to rise overtime while ratings for hybrid students remained essentially 
flat. Teachers’ ratings of students showed not only a significant difference of mean scores 
between distance and hybrid condition (F = 5.76, p<.02), buttheirrate of improvement also 
varied significantly by condition. (F = 6.06, p< .000). The slope of the curve indicates that 
while the trend for distance students was as expected, the ratings for hybrid students 
remained basically flat. fSee Figures 34 and 35.) 

While not statistically significant,”^ the trend for verbal expression of understanding 
also shows the same pattern in Figure 35. This flat trend among hybrid ratings contradicts 
the moderate pre and post-test correlations found between teacher ratings and the pre and 
post test scores. (See Table 6, page 54.) This contradiction may be explained by the com- 
paratively small number of Teacher Logs completed during the experimental period. 

The ratings comparing hybrid and distance students by teachers on Teacher Logs 
were derived differently. Teachers working with distance learners were limited to informa- 
tion about their students that reached them only by telephone conversation and from 
material received by mail and rarely worked face to face with students. Teachers working 

43 

Since the repeated measures program only uses students with a complete set of data, the N’s were small. Only 37 hybrid 
and 21 distance students were included. With increased N*s, the interaction effect might have been significant. 
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in the hybrid mode met their students either singly or in small groups on a regular basis. 
Teachers had the help of reading student body language, responding to facial and hand 
expressions, and developing unique personal relationships with their students. 

Although hybrid cues certainly affected the teachers’ rating processes to some de- 
gree and the absence of these cues played some role for the teachers in the distance 
mode, the comparatively strong correlations between the test scores and the teachers’ 
judgment in both experimental conditions are notable. Teachers working in the distance 
condition may have compensated for the lack of direct cues by recognizing the greater 
improvement among distance learners of their listening skills. 

Correlations between Teacher Evaluation and Test Scores 

To determine the degree of overlap between teacher ratings and test scores, cor- 
relation analyses were run. The agreement between teacher ratings and test scores was 
moderate, but the correlations were statistically significant. (See Figure 36: Coefficient of 
Correiation between Teachers’ Ratings and Pre and Post Test Scores.) 

Table 6. Coefficient of Correlation between Teachers’ Ratings and 
Pre and Post Test Scores 





Understanding the Video 


Ability to Express Understanding in English 


Log 1 / Pre 


Log 5 / Post 


Log 1 / Pre 


Log 5 / Post 


BEST Reading 


0.43 


0.37 


0.39 


0.27 


CASAS Reading 


0.56 


0.77 


0.53 


0.68 


CASAS Listening 


0.61 


0.47 


0.65 


0.40 



The first set of teacher ratings on Log 1 of student understanding of the video and 
the student’s ability to express verbally that understanding were correlated with pre-test 
scores on the BEST and CASAS Reading and Listening respectively. Similarly, the last 
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set of teacher ratings on Log 5 of the student’s understanding of the video and his/her 
ability to express verbally that understanding were correlated with post-test scores on the 
BEST and CASAS Reading and Listening. 

This correlation validated a teacher and test score overlap — teacher ratings and 
tests were on the same track — yet indicated that both were also measuring an unique ele- 
ment of student achievement. The teacher ratings themselves were highly inter-correlated 
(r = .78, p<.000) and may be measuring fairly similar aspects of student understanding. 

The generally moderate correlation between teachers’ independent subjective eval- 
uations and standardized test scores is noteworthy, implying that teacher ratings have util- 
ity in assessing student progress on Crossroads Cafe. 

As both hybrid and distance students’ statistically significant improvement on the 
BEST and CASAS Reading tests correlated comparatively well with teachers’ assess- 
ments, these correlations help to inform the debate about using professional teachers’ 
judgments as opposed to relying entirely upon "objective" assessments. (See Figure 36: 
Coefficient of Correlation between Teachers’ Ratings and Pre and Post Test Scores.) 

Teacher feedback on rating instruments did reveal a degree of discomfort among 
instructors regarding the subjective scales employed. Whether the teachers’ discomfort 
arose from a poorly designed log format or from teachers’ insufficient training in assessing 
student progress, the use of logs is an area worthy of further investigation. 

As discussed in the next section, student logs are, perhaps, even less reliable as 
collectors of experimental data. These logs, though often necessary and perhaps even 
indispensable, require more study and refinement before they can be considered truly 
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reliable research instruments. In this study, data gleaned from student logs were often 
ambiguous. (Please see section "Studying Behaviors," page 59.) 

Student Ratings 

At pre and post-testing sessions, students were asked to rate themselves and their 
own abilities in the following categories: 

■ How comfortable they felt while speaking English on the telephone; 

■ How well they understood English language radio; 

■ How well they understood English language television; and 

■ How much confidence they had in their English language skills. 

Once more a five point scale was used. The number "1" represented the smallest 
degree of comfort, understanding, or confidence through number "5" which represented 
the highest degree of comfort, understanding or confidence. Participants also had the 
option of selecting "I do not speak English on the telephone" or "I do not listen to or watch 
English language radio or television." 

To determine that the four categories of student ratings listed above were evaluating 
unique aspects of the students’ experiences, correlational analyses were run on the post- 
course student ratings of their English language comfort levels. All post-course student 
ratings were found to be inter-correlated, but that inter-correlation was at a sufficiently low 
level as to preclude the likelihood that all four questions were measuring the same skill or 
ability. A Pearson Correlation analysis generated the correlations appearing in Table 7. 

Correlations between the rating scales ranged from r = .32 for comfort on the tele- 
phone and confidence in English language skills to r= .81 for understanding English lan- 
guage radio and understanding English language television. (See Figure 37: Unadjusted 
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Mean Gains in Student Ratings of Telephone Comfort Level; and see Figure 38: Unad- 
justed Mean Gains in Student Ratings of Radio/TV Listening Comfort.) 

Table 7: Multiple Correlations of Perceived Comfort Levels 



Comfort Level Correlations 


English 


English on 


Listening to 


Watching 






Language 


the 


English 


English 






Confidence 


Telephone 


Radio 


Language TV 


Confidence English 


Correlation 


1.000 


*.322 


*.634 


*.615 


Language Skills 


Sig. (2>tailed) 




.000 


.000 


.000 




N 


214 


185 


192 


205 


Speaking English on 


Correlation 


*.322 


1.000 


*.402 


*.355 


the Telephone 


Sig. (2-tailed) 


.000 




.000 


.000 




N 


185 


199 


180 


189 


Listening to English 


Correlation 


*.634 


*.402 


1.000 


*.807 


Language Radio 


Sig. (2-tai!ed) 


.000 


.000 




.000 




N 


192 


180 


207 


199 


Listening to English 


Correlation 


*.615 


*.355 


*.807 


1.000 


Language TV 


Sig. (2-tailed) 


.000 


.000 


.000 






N 


205 


189 


199 


217 



When comparing pre and post-test student ratings, researchers hypothesized that 
distance and hybrid students would emerge with significantly higher ratings on all scales 



as compared to controls. ANCOVA analyses were used to test this hypothesis with initial 
pre-course participant ratings controlled. Distance and hybrid post-test student ratings 
were significantly higher than controls in level of comfort on the telephone (t = 2.56, 
p<.01 1) and in understanding of English language radio (t = 2.14, p<.03). Adjusted post- 
course means of these student comfort level ratings on a scale of 1 to 5 were as follows: 
Comfort on the Phone Comfort Listening to English Language Radio 



Control . . . 2.80 Control 2.59 

Hybrid .... 3.45 Hybrid 2.70 

Distance . . 3.10 Distance 2.85 



The adjusted post-test means of these two rating scales seem to be tapping into 
purely aural understanding. To explore further the lack of significance of the other two 
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scales, "Confidence in English Usage" and "Watching English TV," length of time in the 
study was entered into the model. Not surprisingly, degree of confidence, a variable usual- 
ly resistant to short-term change, increased only for students studying for a full cycle (See 
Figure 39: Post-Test Student Ratings of Confidence in English Usage.) This "Increase in 
Confidence" did not change in different ways across the three experimental conditions. 
(See Figure 40: Students’ Perceived Mean Gains in English Proficiency.) 

As with teacher ratings, correlations between student rating scales and test scores 
were also evaluated. The degree of overlap between student and test measures was 
examined. The analysis revealed little or no correlation between the students’ pre-test 
ratings of their own English language skills with pre-test BEST or pre-test CASAS scores. 
One small statistically significant correlation between level of understanding of English lan- 
guage television and CASAS Listening (r = .23, p<.009) was discovered. 

Small but statistically significant correlations also were found between participants’ 
post-test perceived level of understanding of English language radio and television and the 
post-test CASAS scores. These correlations ranged from r = .22 to r = .38. Confidence 
level in using English was related only to post-test BEST scores with a small correlation 
of r = .24. The data revealed student ratings overlapped only to a small degree with test 
scores. These weak correlations indicate that students seemed to be measuring different 
components of student success rather than those assessed by the BEST and CASAS. 

Finally, correlations between post-test student ratings and teacher ratings were 
perused. The nature of the correlation can help assess the degree to which teachers and 
students are measuring the same perceptions and the same behaviors. 
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Table 8. Correlations between Students’ Self Assessment and Teacher Ratings — Post*Test 





Understanding the Videos 


Ability to Express Verbally that Understanding 


Confidence in English Skills 


.34 


.42 


Comfort on the Phone 


.05* 


.14* 


Understanding English Radio 


.30 


.38 


Understanding English TV 


.25 


.31 


English Improvement 


.35 


.41 



* Not Significant 



Small to moderate statistically significant correlations were found for all possible 
relations, except for the students’ level of comfort on the telephone. (See Table 8.) Sta- 
tistically significant correlations ranging from r =.25 to r =.42 seem to indicate that, although 
the two sets of ratings overlap somewhat, teachers’ and students’ ratings reflect different 
perceptions of student abilities and behaviors. 

Studying Behaviors 
Time on Task 

A common hypothesis at the start of the study was that as students studied more 
(more time-on-task) and as the amount of teacher contact with students increased, more 
learning would occur as measured by the BEST or CASAS Battery and by other mean- 
ingful life style changes. An expectation also was that those with greater initial English 
proficiency would progress more rapidly in their mastery of English than those whose initial 
proficiency in English was less developed. 

To test these hypotheses, several measures to study student behavior were 
included on the student and teacher logs. Among other items, students were asked to 
record on each of their five logs the following data: 
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o The total number of minutes the student watched each video, 

o The total number of minutes the student worked in the Worktext or Photo 
Stories, 

o The number of minutes the student spent working with a partner, and 

o The number of times the student watched the complete video. 

Teacher Logs were used to compile information regarding contact hours. Teachers 
were asked among other items to record: 

o The number of minutes of interaction either in person or by phone between 
the student and teacher that took place for each of the units reported. 

The means of these values were calculated for each of the five categories and a re- 
gression analysis was run for the BEST, for the CASAS Reading, and for the CASAS Lis- 
tening Tests."*^ The analyses, essentially, found that none of these time-on-task variables 
was statistically related to student achievement, except that the mean number of minutes 
spent working with a partner had a negative relationship with the CASAS Listening Test. 
However, in practical terms, the B coefficient was so small as to be negligible. (B = -.014) 

In an effort to explain the absence of a time-on-task effect, the Student Logs were 
reviewed carefully. Evidence surfaced that the Student Logs were probably difficult for 
LEP students to complete. When the data were examined, inconsistencies among the 
responses were obvious. Student Logs reporting multiple viewings of a video often indi- 
cated virtually no minutes spent watching the video. Conversely, on one log, a report of 
eight complete viewings was accompanied by 3500 minutes of watching time. 

In response to the question regarding identification of a partner, students could 
select, "I did not work with a partner this week." If they selected this response, they should 

44 

The forward stepwise selection method was used and the pre-test score controlled for different levels of initial ability. 
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also have logged zero (0) minutes working with a partner. However, many students 
provided discrepant responses for these items. The review revealed that on Student Log 
1 alone, eighteen (18) students reported that they did not work with a partner, but four (4) 
of them had logged significant partner minutes for the same unit. 

Strong correlations greater than .85 would have been expected between the items 
cited which, in effect, are internal checks upon reporting accuracy, such as the number of 
times a complete video was watched and the number of minutes spent watching it. How- 
ever, these correlations were absent. For each student log, correlations between these 
variables ranged from r =.26 and r =.76. 

Trying a different method to uncover correlations, student log reports of time-on-task 
were segregated into three levels. "Low" represented a student spending very little time 
studying; "middle" was average time-on-task; and "high" represented students studying a 
great deal. Even this analysis did not prove fruitful. In fact, in correlation analyses, time- 
on-task variables were unrelated to teacher or student ratings. The data compiled from 
Student Logs must, in almost all situations, remain suspect. 

While not related to student achievement, more confidence can be placed in the ac- 
curacy of teacher contact data. A review of teacher contact minutes'’® revealed that hybrid 
students worked significantly more with their teachers than did distance learners. (F = 
36.13, p<.000) The data disclosed also that for hybrid students, teacher contact minutes 
decreased through the experimental period. (F = 27.4, p<.000) The trend line for hybrid 
students showed a negative slope while the trend line for distance students stayed essen- 



These results are based on those learners with a complete set of the five logs. This may be a special subset of this 
population and not representative. 
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tially flat. (F = 1 0.12, p<.000) (See Figure 41: Mean Student-Teacher Contact Minutes by 
Log.) 

The negative slope in teacher contact minutes for hybrid students might have been 
anticipated. Hybrid students met with their teachers at schools or at other sites in the com- 
munity only at pre-scheduled times, while distance students contacted their instructors by 
telephone at times chosen as mutually convenient to both. As hybrid students gained con- 
fidence with Crossroads Cafe materials and as their English improved, their ability to study 
independently increased; thus, their need fordirect teacher contact would havediminished. 
Therefore, reduced teacher contact for hybrid students may be a manifestation of the in- 
convenience of attending pre-scheduled instructional sessions combined with the student’s 
increased ability to study independently. 

Conversely, the flat slope starting with fewer contact minutes reported on logs for 
distance students may reflect the realistic constraints of contact limited to the telephone. 
If the 275 minutes that hybrid learners on average spent with a teacher at the start, as 
shown in column 1 of Figure 41, represent an approximate optimum period necessary to 
familiarize students with Crossroads, then distance learners might have benefitted as well 
from such an experience. 

However, the opportunity to spend that much time with an instructor on the phone 
is constrained for distance learners by how often one can get to a phone, when a teacher 
can get to a phone, how long one can stay on a phone, and the students’ perceived value 
of the telephone interaction. The average of approximately one hour on the phone with a 
teacher for each unit remained constant throughout and may represent a realistic amount 
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of time a distance learner may expect to work with a teacher in the distance learning mode. 
At best, therefore, the slope for hybrid learners appears to flatten out at about two hours; 
for distance learners, one hour seems to be the operational mean. 

Problems that Interfered with Studying 

For each log, teachers were asked to record problems mentioned by students which 
interfered with studying. (See Figures 42: Problems Interfering with Studying —All Stu- 
dents.) Among the 195 responses, work was the most frequently mentioned problem. 
Other categories selected with some regularity were illness, transportation, lack of time, 
family problems, childcare. Lack of interest and depression were rarely chosen. Distance 
learners cited transportation as a problem once compared to 15 citations by hybrid 
learners. Childcare was noted five times by distance learners as opposed to 12 by hybrid 
students, reflecting the need for hybrid students to travel from home to meet with a 
teacher.'*® Rarer reasons mentioned included a broken VCR, studying for and the need to 
get a driver’s license, need for glasses, inability to understand the accents of the charac- 
ters in the video, moved, had a baby, and travel. 

Choice of Partners and Family Literacy 

For each student log, students were asked: "Did you watch the Video with your 
family? Yes or No." Not unexpectedly, across both learning conditions, a majority of 
students watched the videos with their families. For all five logs, 274 (67.8%) of 404 hybrid 
student responses indicated that students had watched the videos with their families. 
Among distance learning students, the percentage watching with their families was even 

Note that the N’s for distance responses were small, so that only tentative conclusions can be drawn. 
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higher, 113 positive (80.1%) of 141 responses. (See Figure 43: Percent of Students 
Watching Videos with Families by Log, and Figure 44: Total Responses All 5 Logs — 
Percent of Students Watching Videos with Families.) 

On each log, students were asked: "Who was your partner this week? Select as 
many as apply." Of the 360 responses covering all five logs, students selected their 
children, spouses, and other family members as partners 54.2% of the time. Among 
distance learners, children and family members served as partners 55.9% of the time, 
while 49.3% of hybrid students chose children and family members. Hybrid and distance 
learners turned to friends and colleagues at almost the same frequency, 26.7% to 22.5%, 
but both chose children and relatives as partners 54.2% of the of the time. 

When selecting partners with whom to work, the adult LEP students selected their 
own children 41 .9% of the time, far more than any other choice. The next most frequent 
choice for partner, 16.3% of the time, was a relative other than an offspring. (See Figures 
45, 46, and 47 for Choice of Partners: Hybrid, Distance, and Combined.) Clearly, the stu- 
dents’ families played a prominent role in the English acquisition efforts of the LEP adults. 
Based upon these data. Crossroads Cafe should not be overlooked as an important inter- 
vention in the effort to develop full family literacy. 

The fact that so many learners in the experimental conditions sought a group 
learning experience contradicts the orthodox view of non-traditional learners. Those who, 
historically, sought to learn via correspondence or other at-home studies have consistently 
been categorized as "loners, inner or self motivated, and self-sufficient" learners. The 
instructional methods applied were designed to accommodate this "style" of learning. 
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The findings in this study describe students with a different makeup. The high per- 
centage of family viewing and frequent studying with partners indicate that adult ESL learn- 
ers do not fit the pattern of traditional independent learners. Adult LEP students tend to 
establish social conditions for learning despite the absence of a classroom experience. 
They tend to seek out members of their immediate family or others with whom to watch the 
videos and to learn English in a mutually supportive context. 

Although many LEP adults may succeed on their own, the data suggest that 
creating a "virtual classroom" promises greater benefits for them in Crossroads Cafe and 
in similar non-traditional educational settings. Therefore, instructional methods need to be 
devised to create a sense of partnership and belonging, of group effort and activity, of 
mutual help and understanding among LEP adults that offer the social support they 
apparently seek. The advantages of doing so will be measured in reduced attrition, higher 
test scores, and enhanced self-worth. These indications also bode well for the use of 
Crossroads as an important intervention in the development of full family literacy. 
Post-Test Data Collection 

At post-testing, 237 (86.8%) participants completed the Follow-up Background Infor- 
mation Form. Students were asked to report their study activities, assess whether or not 
their mastery of English had improved, and report changes in their employment status. 
They were asked whether or not they would attribute these changes to improved English 
language skills. 

Control students were asked to report on their English language acquisition activities 
during the experimental period. In accordance with our request and with the research 
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protocols, none of them reported attending classes or studying English formally with 
Crossroads Cafe or with any other structured program during the experimental period. 
fSee Figure 2: Learning Activities Prior to Crossroads as Reported by Retained Students.) 

On the Follow-up Background Information Form, distance and hybrid students also 
judged improvement of their English language skills on a five point scale by an adjusted 
average of 1 .4 points for distance learners and .98 for the hybrid group. This perceived 
growth in language proficiency for the experimental groups compares favorably with the 
.30 of a grade perceived by controls for the same period. (See Figure 39 and Figure 40.) 
These differences are statistically significant. (F=12.87, p<.000). Of 237 responding, 
95.9% said they intended to continue studying English. 

The data regarding employment status revealed the following information about the 
hybrid and distance learning adult ESOL program participants: 

Of the 237 respondents, 

• 59.9% (142) reported that they currently had paying jobs. Of these: 
o 29.2% said they had gotten a raise or promotion, and 

• 72.7% of hybrid students attributed their raise or promotion to 
improved English language skills; 

• 91 .7% of distance students attributed their raise or promotion to 
improved English language skills; and 

• 60% of control students attributed their raise or promotion to improved 
English language skills. 

o 19.5% indicated they had gotten a new job. Of these: 

• 77.8% of hybrid students attributed a new job because of improved 
English language skills; 

• 75% of distance students thought they got a new job because of 
improved English language skills; 
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• 63.6% of controls thought they got a new job because of improved 

English language skills; 

Of the 142 reporting that they were employed, 

• 19.5% indicated that they had gotten a new job during the experimental 
period. 

o 81% of the newly employed rated their new job as better than their 
previous job, and 

o 73.9% of the newly employed felt that they had acquired their new jobs 
because of their improved language proficiency. 

When these results are broken down by condition, fSee Figure 48: Employment 

Changes Attributed by Students to Improved English.) 

• Hybrid learners (72.7%) attributed their pay raises to their improved English 
proficiency, 

• Distance learners cited improved English even more consistently (91.7%), 
and 

• Controls felt that their raises were the result of improved English language 
skills only 60% of the time. 

Acquisition of new and better jobs was also more frequently attributed to improved 
English skills by Crossroad learners: 

• Hybrid learners cited improved English 77.8% of the time, 

• Distance learners cited improved English 75% of the time, and 

• Controls cited improved English 63.6% of the time. 

A difference among groups was that both hybrid and distance learners more 
frequently attributed these positive events to improved English language proficiency. 
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Discussion 



This impact study coiripared three groups: control, and two experimental groups, 
hybrid and distance. Following an experimental period of at least 13 weeks, multiple out- 
comes of student success were measured. Hybrid and distance learners studied English 
using only Crossroads Cafe materials; controls did not formally study English either in the 
classroom or with Crossroads Cafe. The performance of hybrid and distance learners on 
standardized tests of English language proficiency and on other pre-defined outcomes was 
compared to performance by controls. 

Just over 57% of those who were originally tested were retained for post-testing. 
Issues contributing to attrition can be grouped into three general areas: 1 ) difficulties that 
developed because of local factors and teacher turnover that were essentially site specific, 
2) programmatic-related concerns, and 3) low pre-test scores and experimental condition.'*^ 
Unfortunately, issues that tended to elevate attrition were seldom isolated and discrete. 
The complex interaction among the contributing factors confounded the identification of 
most direct “cause and effect” relationships. For example, a differential loss of students 
may be attributed in part to a site’s peculiarities and/or to student background character- 
istics. The degree to which either or both cause the loss of a student lies beyond the 
scope of quantitative analysis. In effect, programmatic recommendations emerge from an 
insightful model built upon collected data and upon quantitative methods that reveal salient 
tendencies, likelihoods, and effects. 



In inferential analysis, missing data raises concerns about the “quality’* of the estimates. That is, are they biased or 
unbiased? As long as attrition is only related to pre-test scores and experimental condition, unbiased estimates can still be calculated 
and statistical analyses can be carried out using listwise deletion. If other variables are related to attrition, but not related to the post-test 
scores, again unbiased estimates can still be calculated and statistical analyses can be carried out using listwise deletion. 
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Program Accessibility and Achievement 



In any educational undertaking designed to serve adults, the program must be 
accessible to students, deemed worthy of their time and effort, and provide them a sense 
of achievement. Although these conditions seem outwardly straightfonward, in reality they 
become quite complex. The word "accessible" assumes different meanings for men and 
women, for single and married students, for women with small children, for men with small 
children, for those who are employed, and for those without jobs, as well as for many 
others too diverse to enumerate. Moreover, accessibility is a "sometime thing." What may 
be accessible today may not be so tomorrow. 

Confronting all of the negative influences is the sense of achievement. Adult 
students may be more comfortable with delayed gratification than are children; but, within 
the context of daily negative pressures, evidence of accomplishment and progress rein- 
forces commitment. As for all others, opportunities for immediate, rather than delayed, 
gratification needs to become an integral part of every adult ESOL program. 
Assignment to Condition 

In addition, as far as possible, the participants were randomly assigned to experi- 
mental conditions, and thus the sample is probably representative of the New York State 
LEP population. In fact, this impact study’s sample mirrored a typical, diverse LEP, ESOL 
population according to a study by Rudes and Stempleski.“® 

Study Population 

This Impact Study sample mirrored a typical diverse LEP ESOL population.“® While 
students originated from many different countries, the preponderance were from Hispanic 

Rudes and Stempleski, 1997; See also the Introduction of this Report. 

49 

Rudes and Stempleski, 1997. See also the introduction to this report. 
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backgrounds. Peoples of Asian and Slavic/Eastern European heritage were also well 



represented in the sample. The proportion of representation in our sample reflected those 

from the US Census (1990). 

o More than 60% of the participants were female, 
o Most were between 30 and 49 years old, and 
o Almost half (49.6%) had at least a high school education. 

Members of the group had multiple responsibilities; 

o They had young children in their homes (64.6%); 
o They had other adult relatives living with them (44.8%); and 
o They worked (51 .9%). 

Many participants were not required to speak English in their everyday life. 

o The median number of hours speaking English per day was 2; and 

o More than half (53.9%) indicated they could get by each day without speak- 
ing English at all. 

Outcomes 

Outcomes support the use of Crossroads Cafe in both hybrid and distance learning 
modalities. Crossroads Cafe learners scored significantly higher than control participants 
on several measures of student success: 

• Crossroads Cafe learners improved their English language proficiency com- 
pared to gains made by controls to a statistically significant level as measured 
by the BEST and CASAS Reading tests; 

• Crossroads Cafe learners exhibited higher levels of comfort speaking English 
on the telephone and higher levels of understanding English language radio as 
measured by student ratings; 

• Learners in the distance condition improved their listening comprehension as 
measured by CASAS Listening test; 

• Crossroads Cafe reading gains emerged for learners studying for both a short 
and full cycle while listening gains appeared only for learners studying for a full 
cycle as measured by CASAS Reading and CASAS Listening tests; 
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• CrossfXiads Cafe learners studying for 26 weeks or more improved their listening 
comprehension as measured by CASAS Listening test; and 

• Learners in the distance condition showed improved levels of understanding the 
video and being able to verbally express that understanding as measured by 
teacher ratings. 

Some surprising and perplexing results also came out of the analyses. Urban parti- 
cipants scored more poorly on CASAS Listening than rural or suburban participants. 
Hybrid students did not show improved levels of understanding and expressing that under- 
standing as measured by teacher ratings. No statistical relationship between the time on 
task variables and student achievement was evident. 

As discussed in the "Research History" section of this report, a few studies had 
already been conducted on Crossroads Cafe.^° In all cases, results positively supported 
intervention with Crossroads Cafe. However, those studies reported small sample sizes, 
short durations, limited sampling of the materials and absence of comparison groups. 
Since then, a National Pilot Study®^ was conducted. This study also revealed positive 
outcomes as measured by CASAS Reading and Listening tests. The outcomes discussed 
in this Impact Study support and strengthen the conclusions derived from other studies. 
Given random assignment of participants to experimental condition and comparison 
groups, a recommendation for using Crossroads Cafe program of study as an instructional 
intervention for LEP adults can be made with a high degree of confidence. 

Ultimately, however, the strength of any study rests on the reliability and validity of 
its measuring instruments. Both the BEST and CASAS tests present strengths and weak- 



Johnston, Brzezinski and Stites, 1996; Brzezinski and Leitner, 1996; McLean, 1997. 
Rudes, Zehler, Hopstock and Stephenson, 1999. 
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nesses. Both are criterion-referenced, standardized tests. Both test skills and abilities re- 
quired in everyday life. Both are easy, though time consuming, to administer. Generally, 
teachers gave positive feedback on the tests. 

However, neither test can assess the richness inherent in a program of learning like 
Crossroads Cafe. A more in-depth comparison of the tests would be informative, one that 
provided easy item analyses that would assess actual content and competencies. Both 
tests, written in the 1980's, probably could be updated. 

As assessment is a necessary element in LEP/ESOL programs, further research is 
warranted in this area. While a more appropriate measure for this population of students 
is lacking, researchers at Educational Testing Service have been developing a theory- 
based English language test (TOEFL) by applying theories of communicative competence. 
(Henning & Cascallar, 1992; Chapelleet. al, 1997; Douglas & Smith, 1997; Duran et. al, 
1985). While BEST and CASAS are not academic tests, and should not be, test design 
optimally relies on results of cognitive and field research. 

This study’s results clearly highlighted the problems inherent in collecting logs of 
teacher and student ratings. Ratings offer another valuable way of measuring the con- 
struct of "student success," but much work needs to be done to increase the reliability of 
these measures. 

Conclusions 

The data collected as part of this Crossroads Cafe Impact Study were from two dif- 
ferent sources and were of two distinct types. The first data were “objective” scores gener- 
ated by pre and post standardized tests; the second were responses collected from more 
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“subjective” teacher and student ratings of achievement and impact recorded on pre and 
post program student surveys and Teacher and Student Logs. Data collected from stan- 
dardized tests were more supportive, while self-reported data were relatively less sup- 
portive of Crossroads Cafe’s value as an educational intervention for LEP adults in non- 
traditional settings. 

Standardized Test Results 

Achievement data from the reading components of the BEST and the CASAS Bat- 
tery, as well as Student Performance Level (SPL) analyses, documented significant and 
robust gains in performance by Crossroads Cafe learners when compared with gains by 
controls. Gains in reading achievement in both hybrid and distance conditions as meas- 
ured by two independent measures, the BEST and the CASAS, were found to be suffi- 
ciently large to be of practical significance. 

However, data from the CASAS Listening pre and post tests supported Crossroads 
Cafe only in the distance experimental condition. Crossroads Cafe may be less useful as 
an intervention for improving listening skills generally. However, at several sites, the field 
conditions that were encountered interfered with the students’ ability to hear the spoken 
directions and the accompanying audio tapes. Therefore, conclusions regarding the 
listening variable may be confounded by the quality of the data collected. 

Teacher and Student Ratings 

A respectable body of data generated by the teachers and students support the 
BEST and the CASAS test results. Teacher ratings of students’ understanding and verbal 
expression, students’ assessments of improvements in their own communications and 
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every day life skills, and the students’ reports of changes in their employment all reinforce 
the conclusion that Crossroads Cafe is an effective educational intervention for LEP adults 
under the experimental conditions. Although these findings were neither as definitive nor 
as robust as the test score gains and no discernible pattern appeared favoring one experi- 
mental condition over the other, the reported data did support the Crossroads Cafe inter- 
vention. These teacher and student surveys, while reflecting the imprecision inherent in 
collecting and using self-reported data, were, nonetheless, useful ancillary sources of 
information that reinforced test score findings. 

Implications for Family Literacy 

Finally, analysis of the data revealed extensive use of Crossroads Cafe in learning 
situations involving family members, especially children. An unexpected advantage offered 
by Crossroads Cafe may be its use in family literacy initiatives. 
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Figure 1 

ESOL Teaching Experience 
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Figure 6 

Pre-Tested Students by Age 
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Figure 7 

Pre-Test Distribution by Condition 
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Figure 8 

Retention: Urban, Suburban, Rural 
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Figure 17 

Months at Current Residence 




42 

c 

0 ) 

"D 

3 




(0 

'<5 

Q 




(/) 

O) 

c 



LU 




o o o o o o o 

CM O OO to CM 



s;u9pn)s 10 jaqujriN 



>» 

(0 

o 




(0 

L_ 

Z3 

O 



CN 

CN 

II 




O 

ERIC 



122 - 92 - 123 




in 

CM 

vH 



CO 

OJ 



(M 

n-H 




o 

CM 

0) 

3 

Ui 

■■■ 

u. 



0 

D) 

(0 

3 

O) 

c 

(0 



(0 
■ ■■ 

" 5 ) 

c 

lU 

"O 

0 

0 

0^ 




0 

a 

0 

0 



o 







CD 



ID 






CO 



CNJ 



O 



s;uapn;s JO jequinN 




L. 

0 

Q. 

0 

0 

Q 



(D 

(D 

(D 

Q. 

>> 

(0 

TJ 



II 

0 II 

^ z 



Oi 



o> 



CO 

04 

v-H 



o 



CD 

04 

V~} 



O 

Q. 



O 

E 
>% 
^ o 

CM Q- 

0 E 

3 ^ 

o) 2 

iZ 0 

c 

0 

0 

73 

0 

C 

'E 

0 




\r> 

o> 



CO 

Ci 

^rH 





130 BEST COPY AVAILABLE 



Figure 23 

Reasons Cited for Studying English 
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Figure 27 

CASAS Reading Mean Point Gains 
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Figure 33 

Unadjusted Mean Gains in CASAS Reading Points 

by Initial Proficiency Level 
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Figure 36 

Coefficient of Correlation between Teachers' Ratings 
and Pre and Post Test Scores 




uoi;e|ajJoo ^uaioij^aoo 



CO 

ir> 



h- 

CO 

<D 



O 
> 
o 

£ 

E 
o 

■E O 
O o 



(0 

o 



U) 



a 

- 

3-0 0 
O) fl) I- 
— . 

J- (0 



(0 



o 

(0 

O) 



(0 



0 

■D 

3 

4-* 

(0 



lO 



o 

(0 

o 

(0 



<*1 



CM 

csi 






f ' 






-I 




•;aV 




T— 

ri 




lO 

CO 



CO 



lO 

c\i 



Csl 



s6ui)e^ uea|/\| 



c 

o 

O 



>% 

X 



(D 

O 

c 

(0 

22 

b 



CjO 



(O 

0 

(O 

o 

Q_ 



(O 



£ 

Q_ 



o> 

o> 

II 



o 

CO 



ERIC 




CO 

CO 



re 

0^ 



0) 

"O 

3 

(0 



I 




CO 



lO 

csi 



lO 

Csi 



lO 

CM 

Csi 



s6ui;b^ UBa|/\| 



CM 



o 

o 




Q> 

U 

C 

(Q 

'4-> 

U) 

Q 



Gi 

G> 



II 



CM 

CD 



O 

ERiC 



- 112 - 



LO 

C£> 



ERIC 



0) 

o 

c 

o 

"O 



o 

o 

O O) 
O) (0 
CO O) 

<D .E 



3 (0 



(0 



,S? Qi := 
iZ o) 
c c 

O Ml 

"O 

3 

U) 

c 5 

0 
h- 

1 

’S 

o 

Q. 




o 

CO 



..." -3;j* .-|3 J 

i‘4 ’■ 'ff ^ 






.y 



■■ -f 5;:t 



^«- 

c4 



■p-^« i| 

"'i| --I 

; sfe 



B 

i«- 



CM 

CO 



CO 



O) 



OO 



CO 



CN CM CM 

sBuue^ ueaiAi 



CO 

csi 



to 

CM 



c 

o 

o 



0 

I- 



0 

o 

>» 

o 

-c 

o 

CO 



rji 

CO 



- 113 - 




Figure 40 

Students' Perceived Mean Gains in English Proficiency 
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Figure 45 

Partner Selection by Hybrid Students 
258 Responses 
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Figure 46 

Partner Selection By Distance Learners 
102 Responses 
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Figure 47 

Choice of Partner - All Students 
360 Responses 
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Appendix A 



Protocol Professional Staff Training Meeting 

February 9, 1 998, 9;00 AM — 4:00 PM 
CUNY Graduate Center, 33 West 42"'^ Street, Room 1810 

Committee Members and instructional Staff: 

Please review all of the enclosed documents and forms prior to our training 
session. 



Denny, Verna 
Diones, Ruth 
Flugman, Bert 
Galek, Stan 
Gonzalez, Pat Mooney 
Headley-Walker, Linda 
Miller, Bob 



Miraflores, Nick 
Mule, Marcia 
Siepel, Maria 
Spiegel, Seymour 
Torres, Beth 
Uchitelle, Joan 



As you can see, we have a very full agenda and will be pressed to get all on our 
agenda accomplished in one meeting. Lunch will be provided for all present and will be 
available at the Graduate School Commons on the 18‘^ floor. 



Attendees: 



At least one teacher and administrator from each of the ten experimental sites will 
be in attendance. All present will receive a full set of texts and workbooks. Crossroads 
Cafe Video Unit 1, sample logs, sample testing materials, and a Protocol Manual. 

Agenda: 

1 . Review the Impact Study plan and clarify the Research Questions. 

2. Explain the established protocols that are to be followed in this study by all 
instructors. 

• Hybrid Model — See Research Design 

Issues that we will review are itemized on page 2. 

• Distance Model — See Research Design 

Issues that we will review are itemized on page 2 & 3. 

• Control Group — See Research Design 
Issues that we will review are itemized on page 1. 

3. Review all of the following forms for content, formatting, ease of use, clarity, 
ambiguity. Drafts of all are enclosed in this package. 

• Initial Crossroads Cafe Student Information Form 

• Student Follow-up Information Form 

• Teacher Log Forms 

• Student Log Forms 

• Teacher’s Background Information Form 
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4. General Issues to be reviewed and resolved 

• Distribution of materials 

• Return of forms — Fax vs pre-paid mailers 

• Final site determination and selection of teachers 

• Recruitment and selection of students 

• Honoraria for teachers — differential based upon salaried or non-salaried 
personnel status 

• Stipends for students — how much and how disbursed 

• Toll free number for teacher support 

• Sequential versus random selection of units to be completed by students 

• Identification of the 5 units for Student and Teacher logs 

5. Proposed Orientation Day agenda at each experimental site 

• Introductory remarks by S. Spiegel and R. Diones 

■ Overview of Crossroads Cafe 

■ Explanation of study structure, purpose, method, groupings 

• BEST testing and immediate scoring to select appropriate candidates 

• Discharge and payment to students who score below level 3 and above 
level 6 

• Fill out Student Background Information Forms with teacher assistance 

• Teach students how to use Crossroads Cafe videos, texts, and other 
support materials 

■ Teach students howto complete Student Logs 

■ Distribute materials to students 

■ View video #1 

• Explain procedures for exchange of videos 

• Announce date for post test 

6. Teacher Training 

• Ensure familiarity of teachers with Crossroads Cafe 

• Thorough indoctrination in protocol and procedures 

• Techniques for minimizing attrition in both hybrid & distance 

• Tips for teachers using Crossroads Cafe 

• Discuss student assessment methods in experimental situations 

• Training teachers to use logs and information forms correctly 

• Use of teacher support mechanisms throughout the study 

• Selection of additional teacher trainers 

At the training session, ail teachers will receive : 

■ A full set of the 26 Crossroads Cafe videos, 

■ One copy each of WorktextA and B, 

■ One copy each of Photo Stories A and B, 
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■ One copy each of Teachers Resource Books A and B, 

■ One copy of The Partner Guide, 

■ One copy each of transcripts of the videos, 

■ A copy of the Protocol Manual, and 

■ A complete set of the data collection forms. 

• Ruth Diones will provide an overview of the study. 

• Marcia Mule and Beth Torres will discuss the use the Crossroads Cafe 
materials. 

• Maria Siepel and Joan Uchitelle will share tips on how to minimize 
attrition, anticipate problems, and suggest solutions for use with 
Crossroads Cafe. Other experienced teachers may also offer suggestions 
and ideas for teaching with Crossroads Cafe. 

• The protocols will then be carefully reviewed and interest groups formed: 

■ Marcia and Beth will lead the group planning to use the hybrid model; 

■ Joan and Maria will lead the group planning to use the distance model; 

■ Ruth will discuss the control group characteristics and procedures; 

■ Ruth will discuss student recruitment, group assignment, student 
orientation. 

• Ruth will introduce the logs and information forms. Teachers will learn 
how they should be filled out, distributed, and returned. 

• Ruth will teach everyone how to administer and score the BEST test. 

• Individual teachers will discuss their own immediate concerns and 
schedule both orientation dates at their sites. 

• The Crossroads Cafe office "hotline" can be accessed from anywhere in 
New York State via the toll free # 1-800-472-7882. 

7. Suggestions for conducting the Student Orientation session at each 
experimental site 

• The site teacher will give the students an overview of Crossroads Cafe. 

• Seymour will explain the study structure, purpose, methods, and stipends. 

• The students will take the BEST test. 

• Tests will be scored during the food break. 
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Students scoring below Level III and above Level VI will be discharged. 
They may be referred to other more appropriate classes. 

Students will be randomly selected and assigned to a hybrid, a distance, 
or a control group from a single waiting list. 

The Student Background Information Forms will be filled in with teacher 
assistance. 

Control group students will attend separate orientation sessions, paid their 
stipend, told when to return for post-testing, and then excused. 

All control group members will be promised priority placement in regular 
Crossroads Cafe instructional situations upon completion of the 
experiment. 

The site teachers will introduce the distance and hybrid students to 
Crossroads Cafe materials. 

All materials will be distributed at no cost to the distance and hybrid 
students; 

■ One copy of Worktext, 

■ One copy of Photo Stories, 

■ One copy of the Partner Guide, 

■ One copy of video #1 , 

■ One or more additional video episodes for students to use next, as 
appropriate. 

The students will learn how to use the videos, the materials, and learn 
how to fill in the student logs. 

Everyone will view the first video and become familiar with all of the 
exercises in the Worktext and Photo Stories books. 

The use of the Partner Guide will also be explained and demonstrated. 
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RESEARCH DESIGN 



Appendix B 



The Research Design ofthe Crossroads Cafe Impact Study was approved by the Proto- 
col and the Technical Committees meeting at CASE on February 9 and 18, 1998. Incor- 
porating the suggestions ofthe Committee members, the Design has been re-structured 
to enhance its clarity and to minimize the number of uncontrolled variables among those 
conditions that may affect the study’s outcomes. The protocols for conducting the re- 
search, training the teachers, and preparing the learners have been formulated with these 
goals in mind. 



IMPLEMENTATION PROTOCOLS 



1. Recruitment 

As customary, each site will seek enrollment through the use of newspapers, local 
cable, radio, churches, school districts, libraries, census map, agencies and so on. 

2. Sampling 

After initial screening to eliminate those who score too high or too low on the BEST, 
learners will be assigned to one ofthe experimental groups: distance learning, hybrid, 
or control. 

3. Sites 

Half of the students will be from in NYC sites and half from outside of NYC. Each site 
minimally will contribute students to one experimental group and to the control group. 
To be considered as a site, a minimum of 25 students at that site will be required. Hy- 
brid and distance learning groups will be established both in New York City and at sites 
outside of New York City. Finally, the selection of agencies and learners will be gov- 
erned by site location and by the willingness of agencies and teachers to participate 
within the constraints of the research design: 



N’s 


New York City 


Outside of NYC 


Total 


Hybrid 


63 


63 


126 


Distance 


63 


63 


126 


Control 


63 


63 


126 


Total 


189 


189 


378 
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4. General Protocols and Procedures for All Conditions 

• No students will have prior experience with Crossroads Cafe. 

• Teachers, if at all possible, will have prior experience with Crossroads Cafe. 

• Each teacher will supervise about 25 students. 

• Teachers will keep a log on students’ work. At the end of the study, teachers will 
also fill out a final evaluation form. 

• The students will be assisted in filling out forms. 

• The competency level necessary for inclusion in the program is a BEST score of 
Level III through Level VI. 

• Control and experimental students will be randomly selected from a single waiting 
list. 

5. Control Group Characteristics 

• Participants will have had no exposure to Crossroads Cafe either before or during 
the experimental period. 

• Participants will complete pre and post-tests and the Student Demographic Back- 
ground Information Forms. 

• No logs will be prepared by teachers or students. 

• After testing, filling in the demographic forms, and receiving stipends, control group 
students will be briefed and excused. 

• Control students should not currently be studying English in a classroom situation, 
and each student agrees not to enroll in a class until after the study ends. 

• Control students to be gleaned from each site. 

• Control group students will attend a separate orientation session. 

• Control students will be sent mailings during the experimental period to sustain their 
interest in the study and reminded in advance of the post-test date. 

• Control students will be paid at pre and post-testing. 

• At post-testing students will receive a full set of the Crossroads Cafe materials as 
a gift and be granted priority admission to the next ESOL class. 
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6. Hybrid Group Characteristics 



• Students will study, review, and use Crossroads Cafe videos and support materials 
at home. Students will have face-to-face contact with a teacher once a week to 
converse, discuss Worktext or Photo Story exercises, read aloud, work on pronun- 
ciation, talk about idioms, and write. Class will be primarily an oral interchange. 

• As much as possible, students will meet in groups and work on the same units at 
the weekly sessions. If students have special needs or if they fall behind, teachers 
will work with them individually to help them catch up. 

• Hybrid students will be pre and post-tested, will provide Demographic Information, 
and, at after the course, will also complete a final evaluation form anonymously. 

• Students will attend the orientation session at which video #1 will be shown. 

• Student Logs will be collected for 5 units: 

■ The first student log will be collected when the student completes his/her 3"''unit. 
All students will work on the unit entitled. Worlds Apart, as his/her third unit of 
work. Because students work at various rates, students will be ready to start 
and to complete the 3"^ unit at varying times. 

■ The next student logs will be collected when students complete their 5*^, 7*^, and 
10*^ units. As students may work on units in any order, video episode numbers 
will vary for these collection periods. Also, students will be ready to start these 
units at different times during the experimental period. 

■ The final student log will be collected when a student completes his/her 1 2*^ unit 
of work. For the 1 2*^ unit, all students will work on the episode. Turning Points. 

• Teachers will include the Student Log Sheets with the video tapes when the 5 video 
units identified below are distributed to the students. 

• Students will return their completed logs in postage paid envelopes to the Cross- 
roads Cafe office in NYC or the logs may returned to their teachers who will mail 
them to the NYC office. 

• Teachers will complete a Teacher Log Form for each student following completion 

of the 3"", 7^, 10*^ and 12"’ units . 

• Teachers will call or send letters to students who have been "no shows" for two con- 
secutive weeks. 
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• Teachers and students will not use any supplemental materials except those 
provided by Crossroads Cafe. 

• Teachers and students are encouraged to use all components of Crossroads Cafe, 
including the Teacher’s Resource Book, the Partner Guide, the Worktext, Photo 
Stories, and the videos. Crossroads Cafe assessment materials are excluded. 

• Students will be encouraged: 

■ to meet with teachers one hour a week; 

■ to work at all levels in the Worktext, 

• to watch the video as often as possible; 

■ to schedule a time during their week to study; 

■ to work with a partner; and 

■ to watch the videos with their families. 

• As part of the orientation session, students will be taught how to study using Cross- 
roads Cafe. Teachers will reinforce these skills with the students in both distance 
and hybrid mode throughout the experimental period. 

• A unit usually will be completed every two weeks; however, flexibility of the program 
will permit a student to proceed at his/her own pace. 

• The students will be paid at pre and post-testing sessions and receive a set of the 
Crossroads Cafe materials at no charge. 

7. Distance Learning Group Characteristics 

• Students will study, review, and use Crossroads Cafe videos and support materials 
at home. 

• Students will contact teachers only by telephone. Students may call instructors as 
often as they wish, but they are encouraged to call at least once a week. 

• Teachers may phone their students as often as once a week, but should phone 
each at least once very two weeks. 

• If students have not initiated contact for two consecutive weeks, teachers will phone 
or send letters to these students. 

• The partner should assist the student as he/she works on the units. 

• The partner is encouraged to be present during the student’s phone sessions with 
the instructor. 
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The students are pre and post-tested, provide Demographic Background Informa- 
tion, and, at the end of the course, complete a final evaluation form anonymously. 

Students will attend the orientation session at which video #1 will be shown. 

Student Logs will be collected for 5 units: 

■ The first student log will be collected when the student completes his/her 3"^ unit. 
All students will work on the unit entitled. Worlds Apart, as his/her third unit of 
work. Because students work at various rates, students will be ready to start 
and complete the 3"^ unit at varying times. 

■ The next student logs will be collected when students complete their 5*’^, 7*'^, and 
10*'’ units. As students may work on units in any order, video episode numbers 
will vary for these collection periods. Also, students will be ready to start these 
units at different times during the experimental period. 

■ The final student log will be collected when a student completes his/her 1 2*” unit 
of work. For the 12*'’ unit, all students will work on the episode. Turning Points. 

Teachers will include the Student Log Sheets with the video tapes when the 5 video 
units identified above are distributed to the students. 

Students will return their completed logs in postage paid envelopes to the Cross- 
roads Cafe office in NYC or the logs may returned to their teachers who will mail 
them to the NYC office. 

The teacher will complete a Teacher Log for each student after units 3 and 12 are 
completed and discussed. 

The teacher will also complete a Teacher Log for each student after the student 
completes his/her 5*^ 7*\ and 10**' units (regardless of the actual unit studied). 

Teachers and students will not use any supplemental materials except those pro- 
vided by Crossroads Cafe. 

As much as possible, teachers and students will use all components of Crossroads 
Cafe, including the Teacher’s Resource Book, Partner Guide, Worktext, Photo 
Stories, and the videos. Crossroads Cafe assessment materials will be excluded. 

Students will be encouraged: 

■ to watch the video as often as possible, 

■ to schedule a time during their week to study, 

■ to work at all levels in the Worktext, 

■ to work with a partner, and 

■ to watch the videos with their families. 
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• As part of the orientation session, students will be taught how to study using Cross- 
roads Cafe. Teachers will reinforce these skills with both distance and hybrid mode 
students throughout the experimental period. 

• A unit usually will be completed every two weeks; however, flexibility of the program 
will permit a student to proceed at his/her own pace. 

• The students will be paid at pre and post-testing sessions and receive a set of the 
Crossroads Cafe materials at no charge. 

• Voice mail will be installed for all distance learning teachers. 

• A standardized letter will accompany each video tape as it is distributed to the 
student. 
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Descriptors 



Appendix C 



Proficient Skills 

5PL8** Listening/Speaking: Can participate effectively in social and familiar work situations; can 
understand and participate in practical and social conversations and in technical discussions in own 
field. Reading/Writing: Can handle most reading and writing tasks related to life roles; can read and 
interpret most non-simplified materials; can interpret routine charts, graphs, and labels; fill out 
medical information forms and job applications. Employability: Can meet work demands with 
confidence, in teract with the public, and follow written instructions in work manuals. 

Adult Secondary 

SPL 7 List ening/Spea king: Can function independently in survival and social and work situations; 
can clarify general meaning and communicate on the telephone on familiar topics. Reading/Writing: 
Can read and interpret non-simplified materials on everyday subjects; can interpret routine charts, 
graphs, and labels; fill out medical information forms and job applications; and write an accident or 
incident report. Employability: Understands routine work-related conversations. Can handle work 
that involves following oral and simple written instructions and interact with the public. Can 
perform reading and writing tasks, such as most logs, reports, and forms, with reasonable accuracy to 
meet work needs. ^ ' 

Advanced ESL 

SPL 6 Listening/Speaking : Can satisfy most survival needs and social demands. Has some ability to 
understand and communicate on the telephone on familiar topics. Can participate in conversations on a 
variety of topics. Reading/Writing: Can read and interpret simplified and some non-simplified 

materials on familiar topics. Can interpret simple charts, graphs, and labels; interpret a payroll stub; 
and complete a simple order form; fill out medical information forms and job applications. Can write 
short personal notes and letters and make simple log entries. Employability: Can handle jobs and 
job training situations that- involve following oral and simple Mrritten instructions and multi-step 
diagrams and limited public contact. Can read a simple employee handbook. 

Pers ons at the upper end of this score range are able to begin GED preparation. 

High Intermediate ESL 

SPL 5 Listening/Speaking: Can satisfy basic survival needs and limited social demands; can follow 
oral directions in familiar contexts. Has limited ability to understand on the telephone. Understands 
learned phrases easily and new phrases containing familiar vocabulary. Reading/Writing: Can read and 
interpret simplified and some authentic material on familiar subjects. Can write messages or notes 
related to basic needs. Can fill out basic medical forms and job applications. Employability: Can 

handle jobs and/or training that involve following basic oral and written instructions and diagrams if 
they can be clarified orally. 

Low Intermediate ESL 

SPL 4 Listening/S peaking: Can satisfy basic survival needs and very routine social demands. 

Understands simple learned phrases easily and some new simple phrases containing familiar vocabulary, 
spoken slowly with frequent repetition. Reading/Writing: Can read and interpret simple material on 
familiar topics. Able to read and interpret simple directions, schedules, signs, maps, and menus. Can 
fill out forms requiring basic personal information and write short, simple notes and messages based 
on familiar situations. Employability: Can handle entry-level jobs that involve some simple oral and 
written communi cation but in which tasks can also be demonstrated and/or clarified orally. 

High Beginning ESL 

SPL 3 Listening/Speaking: Functions with some difficulty in situations related to immediate needs; 
may have some simple oral communication abilities using basic learned phrases and sentences. 
Reading/Writing: Reads and writes letters and numbers and a limited number of basic sight words 
and simple phrases related to immediate needs. Can write basic personal information on simplified 
forms. Employability: Can handle routine entry-level jobs that involve only the most basic oral or 
written communicati on in English and in which ah tasks can be demonstrated. 

Low Beginning ESL 

SPL 2 Listening/Speaking: Functions in a very limited way in situations related to immediate needs; 
asks and responds to basic learned phrases spoken slowly and repeated often. ReadingAVriting: 
Recognizes and writes letters and numbers and reads and understands common sight words. Can write 
own name and address. Employability: Can handle only routine entry-level jobs that do not require 
oral or written communi cation in English and in which all tasks are easily demonstrated. 

Pre-Beginning ESL 

SPL 0-1 Listening/Speaking: Functions minimally, if at all, in English. Communicates only through 

language, 
r written 
limited. 
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gestures and a few isolated words. Reading/Writing: May not be literate in any 

Employability: Can handle very routine entry-level jobs that do not require oral oi 

communication in English are easily demonstrated. Employment choices would be extremely 



198 



BESTCOPYAVAIUBLE 



Initial Student Background Information 



student ID Number 



Dear Student, | 

We would like to know more about students participating in Crossroads Cafe. Please 
answer the questions below. Write slowly and carefully in capital letters with letters not 
touching the boxes, and fill in ovals completely. Thank you for your cooperation. 



First Name I. Last Name 



L 


















□ ni 


r _ 
















Number & Street Address 


I 














































City 


















State 


Zip Code 
















I 


















I 


1 1 


I 


I 


I 










- 











How long have you lived at this address? 
Date of Birth: 



/ 



/ 



Month 

Site 



Day 



Year 



Years 



Months 



O Southeni Westchester BOCK 
O Long Beach 



O Ffontier Central 
O Rush Henrietta 
O Shorefront YM-TWHA 
O HighbrMge Community Life Center 
O Chinatown Manpower 
O Bronx Educational S e rvic es 
O Father BHIiiii 
O Flatbush Haitian Center 

Personal 

O Mala 

1) Are you male or female? 

O Female 



2) In which country were 
you bom? 
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Student ID Number 

3) What is your first language? 

(mother tongue) — I I I 

a) Can you read in that language? O Yes b) Can you write in that language? O Yes 

O No O No 



4) How many years of school did you complete? 



5) Do you have any chidren living with you under the age of 18? O Yes 

O No 



6) Are your parents, in-laws or any other adult relatives living with you? 



7) How long have you lived in the 
United States? 







Months 







O Yes 
ONe 



Employment 



1) Do you have a paying job? 



O Yes 
ONo 



2) If so, how many hours do you work each week? 



3) How long have you had this Job? 



Years 







Months 







English Background 

1) What have you done to team English in the past? (Please fill in as many 
ovals as apply.) 

O Studied English using Crossroads Cafe. 

O Attended English-language classes. 

O Studied English-langnage textbooks at home. 

O Listened to English-langnage television or radio. 

O Read English-language newspapers or books. 

O Spoke English with people. BEST COPY AVAILABLE 
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2) Why do you want to study English now? (Please fill in as many ovals as apply.) Student ID Number 

O To become a cHizett of the United States of America. 

O To be able to make more money. 

O To understand my children when they speak English. 

O To be able to help my children more. 

O To be better able to do everyday activities in my neighborhood (For example, to shop.) 

O To get my Graduate Equivalency Degree. 

O To be able to study fbra degree other than GED. 

O Other (Please specify.) 



3) What are you doing now to learn English? (Please fill In as many ovals as apply.) 
O Studying or watching Crossroads Cafe. 

O Attending English-language classes. 

O Studying English-tanguage textbooks at home. 

O Listening to English-tanguage television or radio. 

O Reading English-language newspapers or books. 

O Speaking English with people. 



4) Approximately, how many hours of the day do you have to speak English (For 

example, at work or when shopping?) 

5) Do you live in a community where you could get by without speaking English? O Yes 

O No 

6) Approximately, how many days a week do you read an English 
language newspaper? 




7) How comfbrtable are you speaking on the telephone in English? 
(Please fill in one oval.) 



O very uncom f o rta ble 
O uncomfortable 
O somewhat comfortable 
O comfortable 

0 very comfortable 

0 1 do not speak English on the telephone. 
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B) How much do you understand when you listen to English language radio? 
(Please fill in one oval.) 

O very little 
O a little 
O some 
O most 
Oall 

O I do not listen to English language radio. 



Student ID Number 



9) How much do you understand when you watch English language television? 
(Please fill in one oval.) 

O very little 
O a little 
O some 
O most 
Oall 

O I do not watch English language television. 



10) How confident do you feel when you need to use your English language skills? 
(Please fill in one oval.) 

O not confident at all 

O not confident 
O somewhat confident 
O co n f id e nt 
O very confident 
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Follow-up Student Background Information student id N umber 



Dear Student, 

We would like to know how Crossroads Cafe has affected your life. Please answer the 
questions below. Write slowly and carefully in capital letters with letters not touching 
the boxes, and fill in ovals completely. Thank you for your cooperation. 



First Name 



I. Last Name 

□ 



Number & Street Address 



City 



State 



Zip Code 



How long have you lived at this address? 



Years 







Months 







Date of Birth: 

/ 



/ 



Site 

O Southern Wertcheater BOCES 
O Long Beach 
O Frontier Central 
O Rueh Henrietta 
O Shorofrofit TM-YWHA 
O Hig h bridge Coaiminitty Life Center 
O Chinatown Manpower 
O Bronx Educ a tio na l Services 
O Father BWint 
O FlatiMish Haitian Center 

Employment 

1) Do you have a paying job? O Yes 

O No 

2) If so, how many hours do you work each week? 



Month Day Year 

Yeaeher”s Last Name: 



Hours 



3) How long have you had this Job? 



Years 







Months 







747 



o 
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4) In the last 6 months, did you get a raise or promotion? O Yes 

O No 

If yes, do you think you got the raise or promotion because of your O Yes 
improved English language skills? q 



5) In the last 6 months, did you get a new job? O Yes 

O No 

If yes, do you think this is a better job for you? O Yes 

ONo 

Do you think you got the new job because of your O Yes 

improved English language skills? O No 



English Background 

1) What have you done to learn English in the past 26 weeks? (Please fill in as 
many ovals as apply.) 

O studied or watched Crossroads Cafe. 

O Attended English-language classes. 

O Studied English-ianguage textbooks at home. 

O Listened to English-language television or radio. 

O Read English-language newspapers or books. 

O Spoke English with people. 

2) How comfortable are you speaking on the telephone in English? 

(Please fill in one oval.) 

very uncomfortable 
uncomfortable 
somewhat comfortable 
comfortable 
very comfortable 

I do not speak English on the telephone. 



BEST COPY AVAILABLE 




Student ID Number 
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student ID Numl»er 



3) How much do you understand when you listen to English language radio? 

(Please fill In one oval.) 

O very little 
O a little 
O some 
O most 
Oall 

O I do not listen to English language radio. 

4) How much do you understand when you watch English language television? 

(Please fill in one oval.) 

O very little 
O a little 
O some 
O most 
Oall 

O I do not watch English language television. 

5) How much do you believe your English language skills improved after taking Crossroads 
Cafe? (Please fill in one oval.) 

O Very Little 

O Somewhat 

O Some 

O A Fair Amount 
O ALot 

6) How confident do you feel when you need to use your English language skills? 

(Please fill in one oval.) 

O not confident at all 
O not confident 
O somewhat confident 
O confident 
O very confident 

7) Do you intend to continue studying English? O Yes 

O No 
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Third Unit Studied 



First Name 



Teacher Log 1 



I. Last Name 



Student ID Number 



□ 



(Area Code) Telephone Number 



i\TD) [ 



Student Name 



First Name 



Site 

O Soutlieni Westchester BOCES 
O Long Beach 
O Frontier Central 
O Rush Henrie t t a 
O Shorefrent YM-YWHA 
O Highbridge CdNnmunity Life Center 
O Chinatown Manpower 
O Bronx Educational Services 
O Father BiHini 
O Flatbttsh Haitian Center 



Last Name 



Please record here the date, unit # and amount of time you have spoken with your students about the first, 
second and third units studied. 

Unit #s Pate of contact; Total time in minutes 



/ / 



Month 



Day 



Year 



/ / 



Month Day Year 



/ / 



Month Day Year 



/ / 



Month Day Year 



/ / 



Month Day Year 



/ / 



Month Day Year 



33062 
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Unit «i 



student ID Number 



tn 2 



0 3 



Please rate the student using the scales following the question. If you have any comments, 
write them in the rectangles provided below each question. 



The Video (Fili in oval of best description). 

Rate the student's general understanding of this video. The scale goes from 1 to 5, 
where IsHardiy any SsFair understanding SsExeeilent understanding. 

Ol 02 03 04 05 

Mmments (optional) 



Rate how well the student was able to communicate verbally his/her general 
understanding. 1=Hardly at all SaFair verbal expression 5= excelient verbal expression 

Ol 02 03 04 05 

comments (optionai) 
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student ID Numbc^r 



til s 



While studying this unit, did the students report any problems that interfered 
with studying Cmssroads Cafla? (Please fill in as many ovals as apply.) 

O Illness in the temily 
O Work 

O Transportation 
O Time obligations 
O Lack of interest 
O Depression 
O Family problems 
O Child care 
O Other (please detail.) 



comments (optional) 
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student Log 1 



student ID Number 



Imj 



I 

I 

I 

I 

I 



Dear Student, 

This student log is designed to help you record the amount of time you have worked 
with Crossroads Cafe unit #3, Worlds Apart Each day, please write down the 
approximate amount of time you watched the video, studied the Worktaxt am^ot Photo 
Story and worked with a partner. Also, as you work on this unit, please count the 
number of times you watch the video. After you have finished this unit, answer the 
questions on page 4 and then mail the completed form in the pre-paid envelope 
provided. If necessary, you can return this log sheet to your teacher. Thank you for 
your cooperation. 



First Name 



I. 



Name 



□ 



(Area Code) Teiephone Number 



Teacher^ Last Name: 



I (Lin) 

I 
I 
I 
I 
I 

I 

I 
I 

tXi/ 

Month 

I 

lii 

I o 

ERIC 

r — 



iSite 

I O So u t h e m W estch ester BOCES 
OLoooBmcIi 
I O Ffootier Oeolial 
O Rush Henrietta 
I O Shofefffoirt 

O Hisbbffiflse Co mmuni ty Life Csntsr 

I O Cht nste wm Manpower 
O Bronx Educational Services 
O Fetter BillHii 
|o Flattesb Haitian Center 

Dates that you worked on unit MS: 

!□/□□/[ 

Month 



Unit Numben 

Date of Birth: 

/ 



0 



/ 



Month 



Day 



Year 



Time working on Crossroads Cafe: 
video hours 



Day 



Year 

Worktext & Photo Story hours 
work with partner hours 



and minutes 
and minutes 
and minutes 



/ 



Day 



Year video hours 

Worktext & Photo story hours 
work with partner hours 



and minutes 
and minutes 
and minutes 



25866 
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student ID Number 

sl12 



Dates that you worked on unittfS: 



Time working on Crossroads Cafe: 




Month 



/ 



Month 



/ / 



Day 



/ 



Day 



video hours 

Year 

Worfctext & Photo Story hoars 
work with partner hours 



Year 



video hours 



Worktext & Photo Story hours 



work with partner hours 



and minutes 
and minutes 
and minutes 

and minutes 
and minutes 
and minutes 



Month 



/ / 



Day 



Year 



video hours 



Worktext & Photo Story boars 



work with partner hours 



and minutes 
and minutes 
and minutes 



/ / 



Month 



Day 



Year 



video hours 



Worktext & Photo Story hours 



work with partner hours 



and minutes 
and minutes 
and minutes 



/ 



/ 



Month 



Day 



Year video hours 

Worfctmct & Photo Story hours 
work with partner hours 



and minutes 
and minutes 
and minutes 
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student ID N um b er 

all 3 



Dates that you worked on unit S3: 



Time working on Crossroads Caffe: 



/ 



/ 



Month 



/ 



Month 



/ 



Month 



/ 



Month 



O 

ERIC 



/ 



Month Day 



/ 



Day 



/ 



Day 



/ 



Day 



/ 



Day 



video hours 



Year 

Vfofktext & Photo Story hours 
work with partner hours 



Year 



video hours 



WorMesct & Photo Story hours 



work with partner hours 



Year 



video hours 



Worlctext & Photo Story hours 



work with partner hours 



Year 



video hours 



Worktext & Photo Story hours 



work with partner hours 



Year 



video hours 



Worfctaxt & Photo Story hoars 



work with partner hours 



and minutes 
and minutes 
and minutes 

and minutes 
and minutes 
and minutes 



and minutes 
and minutes 
and minutes 



and minutes 
and minutes 
end minutes 



and minutes 
and minutes 
and minutes 
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Time wofking on Crossroads Cats: 



StiKlairt ID Number 
••I * 



Dates that you worked on Unit #3: 



L 




/ 






Month 


Day 


1 




/ 







Month Day 





|/ 

Month 



_J/ 

Day 



Year vidae howa 

Worfctext & Photo Story hoars 
work with partner hours 



and miniitos 
and miniitos 
and minutes 



Please answer the following questions: 

1) Did you watch the video with your family? OYos 

ONo 

2) Who was your partner this week? (Please fill in as many ovals as apply.) 

O One of my chiMroa 
O A hrfesd or aeishbor 
O A work eeneagae 
O A relative (e.g. brother, sunt,...) 

O Other (Please s p e ci^f.) 

O I dM net work with a partner this week 

3) How many times did you watch the 

complete video? 
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Teacher Background Information 



First Name I. Last Name 























i[ 


□ 


r 
























Number & Street Address 


















































City 




State Zip Code 
























u 


1 


1 












- 











(Area Code) Telephone Number 



(m)[ 



1 ) 



OMale 
O Female 



2) Can you converse in a language 
other than English? 

3) How many years of experience 
have you had teaching English 
to speakers of other 
languages? 



O Yes 
O No 



Years 



Site 

O Mt Vernoo 
O Long Beach 
O Frontier 
O Rush Henrietta 
O Shorefront Y 
O Highbridge 
O Chinatown Manpower 
O Hatbnsh Haitian 
O ELESAIR 

O Bronx Educational Services 



4) How long have you taught using Crossroads CaM 



Years 



Months 



5) Have you taught Crossroads Cafe as a distance 
learning course? 



O Yes 
O No 



6) Have you taught Crossroads Cafe as a "hybrid” O Yes 

learning course? rs 



7) On the average, when you teach using Crossroads 
Cafe, how many students do you work with at the 
same time? 

8) Approximately, how many of hours of training 
have you received In teaching Crossroads Cafe? 
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